2024-08-17T02:09:52.412 INFO:root:teuthology version: 1.2.0 2024-08-17T02:09:52.432 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-17T02:09:52.503 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246 branch: wip-mchangir-mgr-volumes-print-thread-name-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} email: mchangir@redhat.com first_in_suite: false job_id: '7858246' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - filesystem is online with fewer MDS than max_mds - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5995 sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e sleep_before_teardown: 0 suite: fs suite_branch: wip-mchangir-mgr-volumes-print-thread-name-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-17_01:02:08 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-17T02:09:52.503 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa; will attempt to use it 2024-08-17T02:09:52.504 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks 2024-08-17T02:09:52.504 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-17T02:09:52.505 INFO:teuthology.task.internal:Checking packages... 2024-08-17T02:09:52.526 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '2577e8139c6b8786f9d6d6ea21b8da9244b9260e' 2024-08-17T02:09:52.526 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-17T02:09:52.526 INFO:teuthology.packaging:ref: None 2024-08-17T02:09:52.526 INFO:teuthology.packaging:tag: None 2024-08-17T02:09:52.526 INFO:teuthology.packaging:branch: wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-17T02:09:52.526 INFO:teuthology.packaging:sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:09:52.527 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-17T02:09:52.672 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1375-g2577e813-1focal 2024-08-17T02:09:52.674 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-17T02:09:52.680 INFO:teuthology.task.internal:no buildpackages task found 2024-08-17T02:09:52.680 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-17T02:09:52.720 INFO:teuthology.task.internal:Saving configuration 2024-08-17T02:09:52.736 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-17T02:09:52.742 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-17T02:09:52.767 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-17 02:03:46.385727', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-17T02:09:52.786 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-17 02:03:46.384020', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-17T02:09:52.786 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-17T02:09:52.792 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-08-17T02:09:52.792 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-08-17T02:09:52.793 INFO:teuthology.run_tasks:Running task console_log... 2024-08-17T02:09:52.850 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f59e5477ac0>, signals=[15]) 2024-08-17T02:09:52.850 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-17T02:09:52.856 INFO:teuthology.task.internal:Opening connections... 2024-08-17T02:09:52.856 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:09:52.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:09:52.957 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:09:52.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:09:53.044 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-17T02:09:53.050 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-08-17T02:09:53.057 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-08-17T02:09:53.057 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-08-17T02:09:53.106 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu" 2024-08-17T02:09:53.106 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-17T02:09:53.106 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu 2024-08-17T02:09:53.106 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian 2024-08-17T02:09:53.107 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-17T02:09:53.107 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="20.04" 2024-08-17T02:09:53.107 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-17T02:09:53.107 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-17T02:09:53.107 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-17T02:09:53.107 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-17T02:09:53.107 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=focal 2024-08-17T02:09:53.107 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=focal 2024-08-17T02:09:53.108 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-08-17T02:09:53.127 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-08-17T02:09:53.133 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-08-17T02:09:53.134 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-08-17T02:09:53.185 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Ubuntu" 2024-08-17T02:09:53.185 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-17T02:09:53.185 INFO:teuthology.orchestra.run.smithi123.stdout:ID=ubuntu 2024-08-17T02:09:53.185 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE=debian 2024-08-17T02:09:53.186 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-17T02:09:53.186 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="20.04" 2024-08-17T02:09:53.186 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-17T02:09:53.186 INFO:teuthology.orchestra.run.smithi123.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-17T02:09:53.186 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-17T02:09:53.186 INFO:teuthology.orchestra.run.smithi123.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-17T02:09:53.186 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_CODENAME=focal 2024-08-17T02:09:53.186 INFO:teuthology.orchestra.run.smithi123.stdout:UBUNTU_CODENAME=focal 2024-08-17T02:09:53.186 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-08-17T02:09:53.210 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-17T02:09:53.218 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-17T02:09:53.224 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-17T02:09:53.224 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-08-17T02:09:53.226 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-08-17T02:09:53.231 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-17T02:09:53.237 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-17T02:09:53.237 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-08-17T02:09:53.240 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-08-17T02:09:53.290 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-17T02:09:53.346 INFO:teuthology.run_tasks:Running task kernel... 2024-08-17T02:09:53.360 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-17T02:09:53.361 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-17T02:09:53.361 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-17T02:09:53.361 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-17T02:09:53.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:09:53.367 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-08-17T02:09:53.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:09:53.368 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-08-17T02:09:53.374 INFO:teuthology.orchestra.run.smithi100.stdout:5.4.0-124-generic 2024-08-17T02:09:53.374 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.4.0-124-generic 2024-08-17T02:09:53.374 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-08-17T02:09:53.375 INFO:teuthology.orchestra.run.smithi123.stdout:5.4.0-124-generic 2024-08-17T02:09:53.376 INFO:teuthology.task.kernel:Running kernel on smithi123: 5.4.0-124-generic 2024-08-17T02:09:53.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get clean 2024-08-17T02:09:53.478 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-08-17T02:09:53.546 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get update 2024-08-17T02:09:53.640 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-17T02:09:53.641 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-17T02:09:53.641 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-17T02:09:53.705 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-17T02:09:53.714 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-17T02:09:53.721 INFO:teuthology.orchestra.run.smithi123.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-17T02:09:53.786 INFO:teuthology.orchestra.run.smithi123.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-17T02:09:53.821 INFO:teuthology.orchestra.run.smithi123.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-17T02:09:54.015 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,126 kB] 2024-08-17T02:09:54.106 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,017 kB] 2024-08-17T02:09:54.138 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [797 kB] 2024-08-17T02:09:54.150 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-17T02:09:54.150 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,496 kB] 2024-08-17T02:09:54.154 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.2 kB] 2024-08-17T02:09:54.154 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-17T02:09:54.176 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-17T02:09:54.176 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-17T02:09:54.180 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-17T02:09:54.180 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-17T02:09:54.185 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-17T02:09:54.192 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,496 kB] 2024-08-17T02:09:54.193 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-17T02:09:54.195 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.0 kB] 2024-08-17T02:09:54.195 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-17T02:09:54.195 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-17T02:09:54.202 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-17T02:09:54.202 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-17T02:09:54.235 INFO:teuthology.orchestra.run.smithi123.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-17T02:09:54.240 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-17T02:09:54.240 INFO:teuthology.orchestra.run.smithi123.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-17T02:09:54.253 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,017 kB] 2024-08-17T02:09:54.263 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-17T02:09:54.267 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-17T02:09:54.268 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-17T02:09:54.268 INFO:teuthology.orchestra.run.smithi123.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-17T02:09:54.268 INFO:teuthology.orchestra.run.smithi123.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-17T02:09:54.272 INFO:teuthology.orchestra.run.smithi123.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-17T02:09:54.272 INFO:teuthology.orchestra.run.smithi123.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-17T02:09:54.280 INFO:teuthology.orchestra.run.smithi123.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-17T02:09:54.286 INFO:teuthology.orchestra.run.smithi123.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-17T02:09:54.288 INFO:teuthology.orchestra.run.smithi123.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-17T02:09:54.288 INFO:teuthology.orchestra.run.smithi123.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-17T02:09:54.288 INFO:teuthology.orchestra.run.smithi123.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-17T02:09:54.288 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-17T02:09:54.289 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-17T02:09:54.289 INFO:teuthology.orchestra.run.smithi123.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-17T02:09:54.289 INFO:teuthology.orchestra.run.smithi123.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-17T02:09:54.292 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-17T02:09:54.292 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-17T02:09:54.300 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-17T02:09:54.306 INFO:teuthology.orchestra.run.smithi123.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-17T02:09:54.306 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-17T02:09:54.307 INFO:teuthology.orchestra.run.smithi123.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-17T02:09:54.307 INFO:teuthology.orchestra.run.smithi123.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-17T02:09:54.307 INFO:teuthology.orchestra.run.smithi123.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-17T02:09:54.308 INFO:teuthology.orchestra.run.smithi123.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-17T02:09:54.308 INFO:teuthology.orchestra.run.smithi123.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-17T02:09:54.309 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-17T02:09:54.309 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-17T02:09:54.309 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-17T02:09:54.322 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-17T02:09:54.322 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-17T02:09:54.322 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-17T02:09:54.323 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-17T02:09:54.323 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-17T02:09:54.339 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-17T02:09:54.339 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-17T02:09:54.340 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-17T02:09:54.413 INFO:teuthology.orchestra.run.smithi123.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,126 kB] 2024-08-17T02:09:54.897 INFO:teuthology.orchestra.run.smithi123.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [797 kB] 2024-08-17T02:09:54.911 INFO:teuthology.orchestra.run.smithi123.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-17T02:09:54.919 INFO:teuthology.orchestra.run.smithi123.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.2 kB] 2024-08-17T02:09:54.919 INFO:teuthology.orchestra.run.smithi123.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-17T02:09:54.920 INFO:teuthology.orchestra.run.smithi123.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-17T02:09:55.007 INFO:teuthology.orchestra.run.smithi123.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-17T02:09:55.014 INFO:teuthology.orchestra.run.smithi123.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-17T02:09:55.014 INFO:teuthology.orchestra.run.smithi123.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-17T02:09:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-17T02:09:55.075 INFO:teuthology.orchestra.run.smithi123.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-17T02:09:55.079 INFO:teuthology.orchestra.run.smithi123.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.0 kB] 2024-08-17T02:09:55.080 INFO:teuthology.orchestra.run.smithi123.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-17T02:09:55.080 INFO:teuthology.orchestra.run.smithi123.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-17T02:09:55.080 INFO:teuthology.orchestra.run.smithi123.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-17T02:09:55.156 INFO:teuthology.orchestra.run.smithi123.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-17T02:09:58.494 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 21.4 MB in 3s (7,246 kB/s) 2024-08-17T02:09:58.560 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 21.4 MB in 3s (7,252 kB/s) 2024-08-17T02:10:00.053 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-08-17T02:10:00.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-17T02:10:00.113 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2024-08-17T02:10:00.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-17T02:10:00.159 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-08-17T02:10:00.226 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2024-08-17T02:10:00.378 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-08-17T02:10:00.378 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-08-17T02:10:00.440 INFO:teuthology.orchestra.run.smithi123.stdout:Building dependency tree... 2024-08-17T02:10:00.440 INFO:teuthology.orchestra.run.smithi123.stdout:Reading state information... 2024-08-17T02:10:00.573 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-08-17T02:10:00.573 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-17T02:10:00.573 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-17T02:10:00.573 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-17T02:10:00.574 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-08-17T02:10:00.575 INFO:teuthology.orchestra.run.smithi100.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-17T02:10:00.575 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-08-17T02:10:00.575 INFO:teuthology.orchestra.run.smithi100.stdout: thermald 2024-08-17T02:10:00.632 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-08-17T02:10:00.632 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-17T02:10:00.632 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-17T02:10:00.632 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-17T02:10:00.634 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-08-17T02:10:00.634 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-17T02:10:00.641 INFO:teuthology.orchestra.run.smithi123.stdout:The following additional packages will be installed: 2024-08-17T02:10:00.641 INFO:teuthology.orchestra.run.smithi123.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-17T02:10:00.641 INFO:teuthology.orchestra.run.smithi123.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-17T02:10:00.641 INFO:teuthology.orchestra.run.smithi123.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-17T02:10:00.642 INFO:teuthology.orchestra.run.smithi123.stdout:Suggested packages: 2024-08-17T02:10:00.643 INFO:teuthology.orchestra.run.smithi123.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-17T02:10:00.643 INFO:teuthology.orchestra.run.smithi123.stdout:Recommended packages: 2024-08-17T02:10:00.643 INFO:teuthology.orchestra.run.smithi123.stdout: thermald 2024-08-17T02:10:00.708 INFO:teuthology.orchestra.run.smithi123.stdout:The following NEW packages will be installed: 2024-08-17T02:10:00.708 INFO:teuthology.orchestra.run.smithi123.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-17T02:10:00.708 INFO:teuthology.orchestra.run.smithi123.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-17T02:10:00.708 INFO:teuthology.orchestra.run.smithi123.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-17T02:10:00.710 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages will be upgraded: 2024-08-17T02:10:00.710 INFO:teuthology.orchestra.run.smithi123.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-17T02:10:00.818 INFO:teuthology.orchestra.run.smithi100.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-08-17T02:10:00.819 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 77.2 MB of archives. 2024-08-17T02:10:00.819 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-17T02:10:00.819 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-17T02:10:00.893 INFO:teuthology.orchestra.run.smithi123.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-08-17T02:10:00.893 INFO:teuthology.orchestra.run.smithi123.stdout:Need to get 77.2 MB of archives. 2024-08-17T02:10:00.893 INFO:teuthology.orchestra.run.smithi123.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-17T02:10:00.893 INFO:teuthology.orchestra.run.smithi123.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-17T02:10:01.998 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-17T02:10:02.031 INFO:teuthology.orchestra.run.smithi123.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-17T02:10:02.301 INFO:teuthology.orchestra.run.smithi123.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-17T02:10:02.317 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-17T02:10:03.623 INFO:teuthology.orchestra.run.smithi123.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-17T02:10:03.623 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-17T02:10:03.623 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-17T02:10:03.644 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-17T02:10:03.644 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-17T02:10:03.645 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-17T02:10:03.924 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-17T02:10:03.936 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-17T02:10:03.945 INFO:teuthology.orchestra.run.smithi123.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-17T02:10:03.965 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-17T02:10:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 77.2 MB in 3s (23.4 MB/s) 2024-08-17T02:10:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 77.2 MB in 3s (23.8 MB/s) 2024-08-17T02:10:04.820 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-17T02:10:04.881 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-17T02:10:06.546 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-17T02:10:06.551 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-17T02:10:06.590 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-17T02:10:06.595 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-17T02:10:06.650 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:06.711 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:08.542 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-17T02:10:08.559 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-17T02:10:08.602 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-17T02:10:08.618 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-17T02:10:08.665 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:08.717 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:09.745 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-17T02:10:09.758 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-17T02:10:09.795 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:09.797 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-17T02:10:09.811 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-17T02:10:09.847 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:14.715 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-17T02:10:14.747 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-17T02:10:14.867 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-17T02:10:14.874 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-17T02:10:15.201 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-17T02:10:15.232 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-17T02:10:15.345 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-17T02:10:15.362 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-17T02:10:15.673 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-17T02:10:15.688 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-17T02:10:15.689 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-17T02:10:15.705 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-17T02:10:15.731 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-17T02:10:15.747 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-17T02:10:21.057 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-17T02:10:21.079 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-17T02:10:21.107 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-17T02:10:21.123 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:21.128 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-17T02:10:21.165 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:23.857 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-17T02:10:23.984 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-17T02:10:24.008 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-17T02:10:24.152 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-17T02:10:24.324 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-17T02:10:24.456 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:24.486 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-17T02:10:24.632 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:25.675 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:25.850 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:26.819 INFO:teuthology.orchestra.run.smithi100.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-17T02:10:26.819 INFO:teuthology.orchestra.run.smithi100.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-17T02:10:26.960 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:27.010 INFO:teuthology.orchestra.run.smithi123.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-17T02:10:27.010 INFO:teuthology.orchestra.run.smithi123.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-17T02:10:27.086 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:27.152 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:28.679 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-17T02:10:28.814 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-17T02:10:28.940 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-17T02:10:29.066 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:29.133 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-17T02:10:29.133 INFO:teuthology.orchestra.run.smithi100.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-17T02:10:29.297 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-17T02:10:29.414 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-17T02:10:29.540 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-17T02:10:29.686 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-17T02:10:29.777 INFO:teuthology.orchestra.run.smithi123.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-17T02:10:29.777 INFO:teuthology.orchestra.run.smithi123.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-17T02:10:51.188 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-17T02:10:51.188 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub' 2024-08-17T02:10:51.202 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-17T02:10:51.220 INFO:teuthology.orchestra.run.smithi100.stdout:Generating grub configuration file ... 2024-08-17T02:10:51.489 INFO:teuthology.orchestra.run.smithi123.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-17T02:10:51.489 INFO:teuthology.orchestra.run.smithi123.stdout:Sourcing file `/etc/default/grub' 2024-08-17T02:10:51.500 INFO:teuthology.orchestra.run.smithi123.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-17T02:10:51.511 INFO:teuthology.orchestra.run.smithi123.stdout:Generating grub configuration file ... 2024-08-17T02:10:51.546 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-17T02:10:51.566 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-17T02:10:51.858 INFO:teuthology.orchestra.run.smithi123.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-17T02:10:51.882 INFO:teuthology.orchestra.run.smithi123.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-17T02:10:51.929 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-17T02:10:51.941 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-17T02:10:52.287 INFO:teuthology.orchestra.run.smithi123.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-17T02:10:52.298 INFO:teuthology.orchestra.run.smithi123.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-17T02:10:54.666 INFO:teuthology.orchestra.run.smithi100.stdout:done 2024-08-17T02:10:55.007 INFO:teuthology.orchestra.run.smithi123.stdout:done 2024-08-17T02:10:55.076 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 20 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-08-17T02:10:55.123 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.4.0.192.190 2024-08-17T02:10:55.124 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-17T02:10:55.124 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-17T02:10:55.124 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-08-17T02:10:55.124 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-08-17T02:10:55.124 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-08-17T02:10:55.124 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-08-17T02:10:55.124 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-17T02:10:55.124 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-17T02:10:55.124 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-17T02:10:55.125 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-08-17T02:10:55.125 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-08-17T02:10:55.125 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-08-17T02:10:55.242 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-08-17T02:10:55.429 DEBUG:teuthology.orchestra.run.smithi123:> dpkg -s linux-image-generic 2024-08-17T02:10:55.513 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-17T02:10:55.513 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-17T02:10:55.518 INFO:teuthology.orchestra.run.smithi123.stdout:Package: linux-image-generic 2024-08-17T02:10:55.518 INFO:teuthology.orchestra.run.smithi123.stdout:Status: install ok installed 2024-08-17T02:10:55.518 INFO:teuthology.orchestra.run.smithi123.stdout:Priority: optional 2024-08-17T02:10:55.518 INFO:teuthology.orchestra.run.smithi123.stdout:Section: kernel 2024-08-17T02:10:55.518 INFO:teuthology.orchestra.run.smithi123.stdout:Installed-Size: 20 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout:Maintainer: Ubuntu Kernel Team 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: amd64 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout:Source: linux-meta 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout:Version: 5.4.0.192.190 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout:Recommends: thermald 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout:Description: Generic Linux kernel image 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout: This package will always depend on the latest generic kernel image 2024-08-17T02:10:55.519 INFO:teuthology.orchestra.run.smithi123.stdout: available. 2024-08-17T02:10:55.519 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-17T02:10:55.520 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-17T02:10:55.520 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-17T02:10:55.520 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-08-17T02:10:55.520 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=None, version=distro) 2024-08-17T02:10:55.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get clean 2024-08-17T02:10:55.583 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-17T02:10:55.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get update 2024-08-17T02:10:55.665 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-17T02:10:55.728 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-17T02:10:55.734 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-17T02:10:55.752 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-17T02:10:55.846 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-17T02:10:57.372 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-08-17T02:10:57.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-17T02:10:57.466 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-08-17T02:10:57.560 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2024-08-17T02:10:57.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-17T02:10:57.656 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-08-17T02:10:57.657 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-08-17T02:10:57.662 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2024-08-17T02:10:57.877 INFO:teuthology.orchestra.run.smithi123.stdout:Building dependency tree... 2024-08-17T02:10:57.878 INFO:teuthology.orchestra.run.smithi123.stdout:Reading state information... 2024-08-17T02:10:57.882 INFO:teuthology.orchestra.run.smithi100.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-17T02:10:57.882 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-08-17T02:10:57.885 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-08-17T02:10:57.904 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 20 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.4.0.192.190 2024-08-17T02:10:57.905 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-17T02:10:57.906 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-17T02:10:57.906 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-08-17T02:10:57.906 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-08-17T02:10:57.906 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-08-17T02:10:57.906 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-08-17T02:10:57.906 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-08-17T02:10:57.911 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.YYvkzVQFgq 2024-08-17T02:10:57.911 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub/grub.cfg /tmp/tmp.YYvkzVQFgq 2024-08-17T02:10:57.920 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.YYvkzVQFgq 2024-08-17T02:10:58.000 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.YYvkzVQFgq is 9KB 2024-08-17T02:10:58.014 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.YYvkzVQFgq 2024-08-17T02:10:58.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-17T02:10:58.079 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:10:58.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-17T02:10:58.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-17T02:10:58.106 INFO:teuthology.orchestra.run.smithi123.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-17T02:10:58.106 INFO:teuthology.orchestra.run.smithi123.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-08-17T02:10:58.108 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-17T02:10:58.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo update-grub 2024-08-17T02:10:58.109 DEBUG:teuthology.orchestra.run.smithi123:> dpkg -s linux-image-generic 2024-08-17T02:10:58.126 INFO:teuthology.orchestra.run.smithi123.stdout:Package: linux-image-generic 2024-08-17T02:10:58.126 INFO:teuthology.orchestra.run.smithi123.stdout:Status: install ok installed 2024-08-17T02:10:58.126 INFO:teuthology.orchestra.run.smithi123.stdout:Priority: optional 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Section: kernel 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Installed-Size: 20 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Maintainer: Ubuntu Kernel Team 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: amd64 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Source: linux-meta 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Version: 5.4.0.192.190 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Recommends: thermald 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout:Description: Generic Linux kernel image 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: This package will always depend on the latest generic kernel image 2024-08-17T02:10:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: available. 2024-08-17T02:10:58.128 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-08-17T02:10:58.132 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.PmhXpLHger 2024-08-17T02:10:58.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub/grub.cfg /tmp/tmp.PmhXpLHger 2024-08-17T02:10:58.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.PmhXpLHger 2024-08-17T02:10:58.186 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.PmhXpLHger is 9KB 2024-08-17T02:10:58.200 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.PmhXpLHger 2024-08-17T02:10:58.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-17T02:10:58.262 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:10:58.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-17T02:10:58.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-17T02:10:58.292 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-17T02:10:58.292 DEBUG:teuthology.orchestra.run.smithi123:> sudo update-grub 2024-08-17T02:10:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:Sourcing file `/etc/default/grub' 2024-08-17T02:10:59.639 INFO:teuthology.orchestra.run.smithi123.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-17T02:10:59.648 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-08-17T02:10:59.704 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub' 2024-08-17T02:10:59.715 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-17T02:10:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-08-17T02:10:59.938 INFO:teuthology.orchestra.run.smithi123.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-17T02:10:59.949 INFO:teuthology.orchestra.run.smithi123.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-17T02:10:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-17T02:10:59.963 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-17T02:11:00.294 INFO:teuthology.orchestra.run.smithi123.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-17T02:11:00.305 INFO:teuthology.orchestra.run.smithi123.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-17T02:11:00.327 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-17T02:11:00.338 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-17T02:11:01.942 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-08-17T02:11:01.944 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-08-17T02:11:02.031 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-08-17T02:11:02.033 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-08-17T02:11:31.977 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-08-17T02:11:31.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-08-17T02:11:31.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:11:32.036 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-08-17T02:11:32.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-08-17T02:11:32.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:11:50.367 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-08-17T02:11:50.367 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-08-17T02:11:59.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-08-17T02:11:59.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:11:59.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-08-17T02:11:59.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:12:02.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-08-17T02:12:02.431 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-08-17T02:12:14.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-08-17T02:12:14.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:12:14.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-08-17T02:12:14.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:12:32.863 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-08-17T02:12:46.177 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-08-17T02:12:46.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-08-17T02:12:46.766 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-192-generic"... 2024-08-17T02:12:46.767 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-08-17T02:12:46.773 INFO:teuthology.orchestra.run.smithi123.stdout:5.4.0-192-generic 2024-08-17T02:12:46.773 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-17T02:12:46.774 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-17T02:12:46.774 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-17T02:12:47.774 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-08-17T02:12:47.775 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-17T02:12:47.854 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2024-08-17T02:12:47.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-08-17T02:12:47.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:12:47.873 DEBUG:teuthology.parallel:result is None 2024-08-17T02:12:48.100 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-08-17T02:12:48.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-08-17T02:12:48.728 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-192-generic"... 2024-08-17T02:12:48.729 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-08-17T02:12:48.735 INFO:teuthology.orchestra.run.smithi100.stdout:5.4.0-192-generic 2024-08-17T02:12:48.735 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-17T02:12:48.735 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-17T02:12:48.736 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-17T02:12:49.736 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-17T02:12:49.737 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-17T02:12:49.830 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-08-17T02:12:49.856 DEBUG:teuthology.parallel:result is None 2024-08-17T02:12:49.856 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-17T02:12:49.864 INFO:teuthology.task.internal:Creating test directory... 2024-08-17T02:12:49.864 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-17T02:12:49.867 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-17T02:12:49.874 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-17T02:12:49.880 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-17T02:12:49.890 INFO:teuthology.task.internal:Creating archive directory... 2024-08-17T02:12:49.890 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-17T02:12:49.915 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-17T02:12:49.937 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-17T02:12:49.943 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-17T02:12:49.943 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-17T02:12:49.946 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-17T02:12:49.961 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-17T02:12:49.973 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-17T02:12:49.990 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-17T02:12:50.000 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-17T02:12:50.002 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-17T02:12:50.008 INFO:teuthology.task.internal:Configuring sudo... 2024-08-17T02:12:50.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-17T02:12:50.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-17T02:12:50.035 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-17T02:12:50.044 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-17T02:12:50.044 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-17T02:12:50.046 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-17T02:12:50.052 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-17T02:12:50.058 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-17T02:12:50.108 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:12:50.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-17T02:12:50.132 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-17T02:12:50.139 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-17T02:12:50.146 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:12:50.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-17T02:12:50.165 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-08-17T02:12:50.168 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-08-17T02:12:50.215 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-17T02:12:50.221 INFO:teuthology.task.internal:Starting timer... 2024-08-17T02:12:50.222 INFO:teuthology.run_tasks:Running task pcp... 2024-08-17T02:12:50.233 INFO:teuthology.run_tasks:Running task selinux... 2024-08-17T02:12:50.240 DEBUG:teuthology.task.selinux:Excluding smithi100: OS 'ubuntu' does not support SELinux 2024-08-17T02:12:50.240 DEBUG:teuthology.task.selinux:Excluding smithi123: OS 'ubuntu' does not support SELinux 2024-08-17T02:12:50.240 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-17T02:12:50.241 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-08-17T02:12:50.241 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-17T02:12:50.241 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-17T02:12:50.249 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-17T02:12:50.262 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-17T02:12:50.263 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-17T02:18:23.313 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2024-08-17T02:18:23.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-08-17T02:18:23.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:18:23.419 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-08-17T02:18:23.544 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-08-17T02:18:23.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-08-17T02:18:23.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:18:23.647 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-08-17T02:18:23.775 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-08-17T02:18:23.775 INFO:teuthology.run_tasks:Running task clock... 2024-08-17T02:18:23.787 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-17T02:18:23.788 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-17T02:18:23.789 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-17T02:18:23.793 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-17T02:18:23.793 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-17T02:18:23.827 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-17T02:18:23.828 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: Command line: ntpd -gq 2024-08-17T02:18:23.828 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: proto: precision = 0.093 usec (-23) 2024-08-17T02:18:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-17T02:18:23.828 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-17T02:18:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-17T02:18:23.830 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: restrict ::: KOD does nothing without LIMITED. 2024-08-17T02:18:23.830 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: Listen and drop on 0 v6wildcard [::]:123 2024-08-17T02:18:23.830 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-17T02:18:23.830 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: Listen normally on 2 lo 127.0.0.1:123 2024-08-17T02:18:23.830 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: Listen normally on 3 enp3s0f1 172.21.15.100:123 2024-08-17T02:18:23.830 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: Listen normally on 4 lo [::1]:123 2024-08-17T02:18:23.830 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d733%5]:123 2024-08-17T02:18:23.830 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:23 ntpd[13639]: Listening on routing socket on fd #22 for interface updates 2024-08-17T02:18:23.853 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-17T02:18:23.853 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: Command line: ntpd -gq 2024-08-17T02:18:23.854 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: proto: precision = 0.168 usec (-22) 2024-08-17T02:18:23.855 INFO:teuthology.orchestra.run.smithi123.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-17T02:18:23.855 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-17T02:18:23.855 INFO:teuthology.orchestra.run.smithi123.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-17T02:18:23.856 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: restrict ::: KOD does nothing without LIMITED. 2024-08-17T02:18:23.856 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: Listen and drop on 0 v6wildcard [::]:123 2024-08-17T02:18:23.857 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-17T02:18:23.857 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: Listen normally on 2 lo 127.0.0.1:123 2024-08-17T02:18:23.857 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: Listen normally on 3 enp3s0f1 172.21.15.123:123 2024-08-17T02:18:23.857 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: Listen normally on 4 lo [::1]:123 2024-08-17T02:18:23.857 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d3f1%5]:123 2024-08-17T02:18:23.857 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:23 ntpd[13592]: Listening on routing socket on fd #22 for interface updates 2024-08-17T02:18:24.824 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:24.824 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:24 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:24.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:24.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:24.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:24 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:26.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:26.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:26 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:26.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:26.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:26.857 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:26 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:28.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:28.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:28 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:28.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:28 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:30.824 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.824 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.824 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.824 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:30.825 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:30.825 INFO:teuthology.orchestra.run.smithi100.stdout:17 Aug 02:18:30 ntpd[13639]: ntpd: time slew -0.000502 s 2024-08-17T02:18:30.825 INFO:teuthology.orchestra.run.smithi100.stdout:ntpd: time slew -0.000502s 2024-08-17T02:18:30.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-17T02:18:30.826 INFO:teuthology.orchestra.run.smithi100.stderr:17 Aug 02:18:30 ntpd[13639]: can't open /var/log/ntpstats/loopstats.20240817: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-17T02:18:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: can't open /var/log/ntpstats/rawstats.20240817: Permission denied 2024-08-17T02:18:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-17T02:18:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: can't open /var/log/ntpstats/peerstats.20240817: Permission denied 2024-08-17T02:18:30.856 INFO:teuthology.orchestra.run.smithi123.stdout:17 Aug 02:18:30 ntpd[13592]: ntpd: time slew -0.000026 s 2024-08-17T02:18:30.856 INFO:teuthology.orchestra.run.smithi123.stdout:ntpd: time slew -0.000026s 2024-08-17T02:18:30.857 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-17T02:18:30.857 INFO:teuthology.orchestra.run.smithi123.stderr:17 Aug 02:18:30 ntpd[13592]: can't open /var/log/ntpstats/loopstats.20240817: Permission denied 2024-08-17T02:18:30.892 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-08-17T02:18:30.892 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-08-17T02:18:30.892 INFO:teuthology.orchestra.run.smithi100.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:18:30.893 INFO:teuthology.orchestra.run.smithi100.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:18:30.893 INFO:teuthology.orchestra.run.smithi100.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:18:30.893 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:18:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: remote refid st t when poll reach delay offset jitter 2024-08-17T02:18:30.927 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================== 2024-08-17T02:18:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:18:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:18:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:18:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:18:30.928 INFO:teuthology.run_tasks:Running task install... 2024-08-17T02:18:30.938 DEBUG:teuthology.task.install:project ceph 2024-08-17T02:18:30.938 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}} 2024-08-17T02:18:30.938 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'} 2024-08-17T02:18:30.939 INFO:teuthology.task.install:Using flavor: default 2024-08-17T02:18:30.950 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-17T02:18:30.950 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-08-17T02:18:30.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-key list | grep Ceph 2024-08-17T02:18:30.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-key list | grep Ceph 2024-08-17T02:18:31.052 INFO:teuthology.orchestra.run.smithi123.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-17T02:18:31.061 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-17T02:18:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-17T02:18:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:uid [ unknown] Ceph.com (release key) 2024-08-17T02:18:31.112 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-17T02:18:31.112 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-17T02:18:31.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:18:31.119 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-17T02:18:31.120 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph.com (release key) 2024-08-17T02:18:31.121 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-17T02:18:31.121 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-17T02:18:31.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:18:31.265 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default/ 2024-08-17T02:18:31.266 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-g2577e813-1focal 2024-08-17T02:18:31.274 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default/ 2024-08-17T02:18:31.275 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-g2577e813-1focal 2024-08-17T02:18:31.382 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:18:31.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-17T02:18:31.391 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:18:31.391 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-17T02:18:31.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get update 2024-08-17T02:18:31.408 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-08-17T02:18:31.548 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-17T02:18:31.548 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-17T02:18:31.555 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-17T02:18:31.571 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-17T02:18:31.674 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-17T02:18:31.674 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-17T02:18:31.694 INFO:teuthology.orchestra.run.smithi123.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal InRelease 2024-08-17T02:18:31.708 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal InRelease 2024-08-17T02:18:31.748 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-17T02:18:31.749 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-17T02:18:31.765 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-17T02:18:31.809 INFO:teuthology.orchestra.run.smithi123.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal Release.gpg 2024-08-17T02:18:31.825 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal Release.gpg 2024-08-17T02:18:31.825 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-17T02:18:31.871 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-17T02:18:31.888 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-17T02:18:31.947 INFO:teuthology.orchestra.run.smithi123.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main i386 Packages [3,382 B] 2024-08-17T02:18:31.966 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main i386 Packages [3,382 B] 2024-08-17T02:18:32.202 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 28.5 kB in 1s (45.4 kB/s) 2024-08-17T02:18:32.329 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 28.5 kB in 1s (37.3 kB/s) 2024-08-17T02:18:33.349 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-08-17T02:18:33.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-g2577e813-1focal cephadm=17.2.7-1375-g2577e813-1focal ceph-mds=17.2.7-1375-g2577e813-1focal ceph-mgr=17.2.7-1375-g2577e813-1focal ceph-common=17.2.7-1375-g2577e813-1focal ceph-fuse=17.2.7-1375-g2577e813-1focal ceph-test=17.2.7-1375-g2577e813-1focal ceph-volume=17.2.7-1375-g2577e813-1focal radosgw=17.2.7-1375-g2577e813-1focal python3-rados=17.2.7-1375-g2577e813-1focal python3-rgw=17.2.7-1375-g2577e813-1focal python3-cephfs=17.2.7-1375-g2577e813-1focal python3-rbd=17.2.7-1375-g2577e813-1focal libcephfs2=17.2.7-1375-g2577e813-1focal libcephfs-dev=17.2.7-1375-g2577e813-1focal librados2=17.2.7-1375-g2577e813-1focal librbd1=17.2.7-1375-g2577e813-1focal rbd-fuse=17.2.7-1375-g2577e813-1focal python3-cephfs=17.2.7-1375-g2577e813-1focal cephfs-shell=17.2.7-1375-g2577e813-1focal cephfs-top=17.2.7-1375-g2577e813-1focal cephfs-mirror=17.2.7-1375-g2577e813-1focal 2024-08-17T02:18:33.446 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-08-17T02:18:33.470 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2024-08-17T02:18:33.496 DEBUG:teuthology.orchestra.run.smithi123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-g2577e813-1focal cephadm=17.2.7-1375-g2577e813-1focal ceph-mds=17.2.7-1375-g2577e813-1focal ceph-mgr=17.2.7-1375-g2577e813-1focal ceph-common=17.2.7-1375-g2577e813-1focal ceph-fuse=17.2.7-1375-g2577e813-1focal ceph-test=17.2.7-1375-g2577e813-1focal ceph-volume=17.2.7-1375-g2577e813-1focal radosgw=17.2.7-1375-g2577e813-1focal python3-rados=17.2.7-1375-g2577e813-1focal python3-rgw=17.2.7-1375-g2577e813-1focal python3-cephfs=17.2.7-1375-g2577e813-1focal python3-rbd=17.2.7-1375-g2577e813-1focal libcephfs2=17.2.7-1375-g2577e813-1focal libcephfs-dev=17.2.7-1375-g2577e813-1focal librados2=17.2.7-1375-g2577e813-1focal librbd1=17.2.7-1375-g2577e813-1focal rbd-fuse=17.2.7-1375-g2577e813-1focal python3-cephfs=17.2.7-1375-g2577e813-1focal cephfs-shell=17.2.7-1375-g2577e813-1focal cephfs-top=17.2.7-1375-g2577e813-1focal cephfs-mirror=17.2.7-1375-g2577e813-1focal 2024-08-17T02:18:33.577 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2024-08-17T02:18:33.659 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-08-17T02:18:33.659 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-08-17T02:18:33.787 INFO:teuthology.orchestra.run.smithi123.stdout:Building dependency tree... 2024-08-17T02:18:33.788 INFO:teuthology.orchestra.run.smithi123.stdout:Reading state information... 2024-08-17T02:18:33.862 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-08-17T02:18:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-17T02:18:33.863 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-17T02:18:33.863 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-08-17T02:18:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-17T02:18:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-17T02:18:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-17T02:18:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-17T02:18:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-17T02:18:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-17T02:18:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-17T02:18:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-17T02:18:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-17T02:18:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-17T02:18:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-17T02:18:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-17T02:18:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-17T02:18:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-17T02:18:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-17T02:18:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-17T02:18:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-17T02:18:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-17T02:18:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-17T02:18:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-17T02:18:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-17T02:18:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-17T02:18:33.868 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-08-17T02:18:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-17T02:18:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-17T02:18:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-17T02:18:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-17T02:18:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-17T02:18:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-17T02:18:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-17T02:18:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-17T02:18:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-17T02:18:33.870 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-08-17T02:18:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: btrfs-tools 2024-08-17T02:18:33.974 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages were automatically installed and are no longer required: 2024-08-17T02:18:33.976 INFO:teuthology.orchestra.run.smithi123.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-17T02:18:33.976 INFO:teuthology.orchestra.run.smithi123.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-17T02:18:33.976 INFO:teuthology.orchestra.run.smithi123.stdout:The following additional packages will be installed: 2024-08-17T02:18:33.976 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-17T02:18:33.976 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-17T02:18:33.976 INFO:teuthology.orchestra.run.smithi123.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-17T02:18:33.976 INFO:teuthology.orchestra.run.smithi123.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-17T02:18:33.976 INFO:teuthology.orchestra.run.smithi123.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-17T02:18:33.978 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-17T02:18:33.978 INFO:teuthology.orchestra.run.smithi123.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-17T02:18:33.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-17T02:18:33.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-17T02:18:33.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-17T02:18:33.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout:Suggested packages: 2024-08-17T02:18:33.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout:Recommended packages: 2024-08-17T02:18:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: btrfs-tools 2024-08-17T02:18:33.988 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-08-17T02:18:33.989 INFO:teuthology.orchestra.run.smithi100.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-17T02:18:33.989 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-17T02:18:33.989 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-17T02:18:33.989 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-17T02:18:33.989 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-17T02:18:33.989 INFO:teuthology.orchestra.run.smithi100.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-17T02:18:33.989 INFO:teuthology.orchestra.run.smithi100.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-17T02:18:33.990 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-17T02:18:33.990 INFO:teuthology.orchestra.run.smithi100.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-17T02:18:33.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-17T02:18:33.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-17T02:18:33.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-17T02:18:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-17T02:18:33.992 INFO:teuthology.orchestra.run.smithi100.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-17T02:18:33.992 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-17T02:18:33.992 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-17T02:18:33.992 INFO:teuthology.orchestra.run.smithi100.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-17T02:18:33.993 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-08-17T02:18:33.993 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 librbd1 2024-08-17T02:18:34.057 INFO:teuthology.orchestra.run.smithi100.stdout:2 upgraded, 114 newly installed, 0 to remove and 295 not upgraded. 2024-08-17T02:18:34.058 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 201 MB of archives. 2024-08-17T02:18:34.058 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-17T02:18:34.058 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-17T02:18:34.077 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-17T02:18:34.087 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-17T02:18:34.105 INFO:teuthology.orchestra.run.smithi123.stdout:The following NEW packages will be installed: 2024-08-17T02:18:34.105 INFO:teuthology.orchestra.run.smithi123.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-17T02:18:34.105 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-17T02:18:34.105 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-17T02:18:34.105 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-17T02:18:34.105 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-17T02:18:34.106 INFO:teuthology.orchestra.run.smithi123.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-17T02:18:34.106 INFO:teuthology.orchestra.run.smithi123.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-17T02:18:34.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-17T02:18:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-17T02:18:34.115 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages will be upgraded: 2024-08-17T02:18:34.116 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 librbd1 2024-08-17T02:18:34.117 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-17T02:18:34.140 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-17T02:18:34.142 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-17T02:18:34.152 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-17T02:18:34.154 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-17T02:18:34.172 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-17T02:18:34.174 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-17T02:18:34.178 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-17T02:18:34.206 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-17T02:18:34.207 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-17T02:18:34.216 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-17T02:18:34.216 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-17T02:18:34.216 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-17T02:18:34.217 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-17T02:18:34.218 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-17T02:18:34.218 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-17T02:18:34.219 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-17T02:18:34.230 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-17T02:18:34.231 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-17T02:18:34.247 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-17T02:18:34.247 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-17T02:18:34.248 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-17T02:18:34.250 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-17T02:18:34.250 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-17T02:18:34.251 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-17T02:18:34.253 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-17T02:18:34.254 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-17T02:18:34.264 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-17T02:18:34.265 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-17T02:18:34.283 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-17T02:18:34.288 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-17T02:18:34.289 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-17T02:18:34.289 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-17T02:18:34.290 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-17T02:18:34.293 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-17T02:18:34.293 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-17T02:18:34.294 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-17T02:18:34.299 INFO:teuthology.orchestra.run.smithi123.stdout:2 upgraded, 114 newly installed, 0 to remove and 295 not upgraded. 2024-08-17T02:18:34.299 INFO:teuthology.orchestra.run.smithi123.stdout:Need to get 201 MB of archives. 2024-08-17T02:18:34.299 INFO:teuthology.orchestra.run.smithi123.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-17T02:18:34.299 INFO:teuthology.orchestra.run.smithi123.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-17T02:18:34.300 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-17T02:18:34.301 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-17T02:18:34.317 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-17T02:18:34.325 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-17T02:18:34.326 INFO:teuthology.orchestra.run.smithi100.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-17T02:18:34.326 INFO:teuthology.orchestra.run.smithi100.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-17T02:18:34.383 INFO:teuthology.orchestra.run.smithi123.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-17T02:18:34.400 INFO:teuthology.orchestra.run.smithi100.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-17T02:18:34.416 INFO:teuthology.orchestra.run.smithi100.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-17T02:18:34.417 INFO:teuthology.orchestra.run.smithi123.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-17T02:18:34.418 INFO:teuthology.orchestra.run.smithi100.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-17T02:18:34.428 INFO:teuthology.orchestra.run.smithi100.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-17T02:18:34.428 INFO:teuthology.orchestra.run.smithi100.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-17T02:18:34.490 INFO:teuthology.orchestra.run.smithi100.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-17T02:18:34.491 INFO:teuthology.orchestra.run.smithi100.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-17T02:18:34.492 INFO:teuthology.orchestra.run.smithi100.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-17T02:18:34.494 INFO:teuthology.orchestra.run.smithi100.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-17T02:18:34.494 INFO:teuthology.orchestra.run.smithi100.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-17T02:18:34.496 INFO:teuthology.orchestra.run.smithi100.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-17T02:18:34.498 INFO:teuthology.orchestra.run.smithi100.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-17T02:18:34.499 INFO:teuthology.orchestra.run.smithi100.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-17T02:18:34.504 INFO:teuthology.orchestra.run.smithi100.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-17T02:18:34.509 INFO:teuthology.orchestra.run.smithi100.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-17T02:18:34.510 INFO:teuthology.orchestra.run.smithi100.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-17T02:18:34.527 INFO:teuthology.orchestra.run.smithi100.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-17T02:18:34.530 INFO:teuthology.orchestra.run.smithi100.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-17T02:18:34.533 INFO:teuthology.orchestra.run.smithi100.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-17T02:18:34.534 INFO:teuthology.orchestra.run.smithi100.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-17T02:18:34.542 INFO:teuthology.orchestra.run.smithi123.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-17T02:18:34.552 INFO:teuthology.orchestra.run.smithi100.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-17T02:18:34.554 INFO:teuthology.orchestra.run.smithi100.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-17T02:18:34.590 INFO:teuthology.orchestra.run.smithi100.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-17T02:18:34.591 INFO:teuthology.orchestra.run.smithi100.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-17T02:18:34.592 INFO:teuthology.orchestra.run.smithi100.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-17T02:18:34.592 INFO:teuthology.orchestra.run.smithi100.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-17T02:18:34.592 INFO:teuthology.orchestra.run.smithi100.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-17T02:18:34.593 INFO:teuthology.orchestra.run.smithi100.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-17T02:18:34.604 INFO:teuthology.orchestra.run.smithi100.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-17T02:18:34.604 INFO:teuthology.orchestra.run.smithi100.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-17T02:18:34.605 INFO:teuthology.orchestra.run.smithi100.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-17T02:18:34.608 INFO:teuthology.orchestra.run.smithi100.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-17T02:18:34.625 INFO:teuthology.orchestra.run.smithi100.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-17T02:18:34.626 INFO:teuthology.orchestra.run.smithi100.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-17T02:18:34.629 INFO:teuthology.orchestra.run.smithi100.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-17T02:18:34.631 INFO:teuthology.orchestra.run.smithi100.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-17T02:18:34.672 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-17T02:18:34.680 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-17T02:18:34.724 INFO:teuthology.orchestra.run.smithi123.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-17T02:18:34.736 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-17T02:18:34.815 INFO:teuthology.orchestra.run.smithi123.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-17T02:18:34.820 INFO:teuthology.orchestra.run.smithi123.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-17T02:18:34.843 INFO:teuthology.orchestra.run.smithi123.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-17T02:18:34.971 INFO:teuthology.orchestra.run.smithi123.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-17T02:18:34.978 INFO:teuthology.orchestra.run.smithi123.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-17T02:18:35.000 INFO:teuthology.orchestra.run.smithi123.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-17T02:18:35.007 INFO:teuthology.orchestra.run.smithi123.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-17T02:18:35.010 INFO:teuthology.orchestra.run.smithi123.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-17T02:18:35.018 INFO:teuthology.orchestra.run.smithi123.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-17T02:18:35.018 INFO:teuthology.orchestra.run.smithi123.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-17T02:18:35.018 INFO:teuthology.orchestra.run.smithi123.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-17T02:18:35.018 INFO:teuthology.orchestra.run.smithi123.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-17T02:18:35.079 INFO:teuthology.orchestra.run.smithi123.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-17T02:18:35.079 INFO:teuthology.orchestra.run.smithi123.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-17T02:18:35.160 INFO:teuthology.orchestra.run.smithi123.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-17T02:18:35.161 INFO:teuthology.orchestra.run.smithi123.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-17T02:18:35.161 INFO:teuthology.orchestra.run.smithi123.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-17T02:18:35.163 INFO:teuthology.orchestra.run.smithi123.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-17T02:18:35.164 INFO:teuthology.orchestra.run.smithi123.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-17T02:18:35.164 INFO:teuthology.orchestra.run.smithi123.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-17T02:18:35.166 INFO:teuthology.orchestra.run.smithi123.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-17T02:18:35.167 INFO:teuthology.orchestra.run.smithi123.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-17T02:18:35.241 INFO:teuthology.orchestra.run.smithi123.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-17T02:18:35.245 INFO:teuthology.orchestra.run.smithi123.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-17T02:18:35.324 INFO:teuthology.orchestra.run.smithi123.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-17T02:18:35.331 INFO:teuthology.orchestra.run.smithi123.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-17T02:18:35.331 INFO:teuthology.orchestra.run.smithi123.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-17T02:18:35.332 INFO:teuthology.orchestra.run.smithi123.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-17T02:18:35.332 INFO:teuthology.orchestra.run.smithi123.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-17T02:18:35.333 INFO:teuthology.orchestra.run.smithi123.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-17T02:18:35.336 INFO:teuthology.orchestra.run.smithi123.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-17T02:18:35.337 INFO:teuthology.orchestra.run.smithi123.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-17T02:18:35.381 INFO:teuthology.orchestra.run.smithi100.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-g2577e813-1focal [3,506 kB] 2024-08-17T02:18:35.387 INFO:teuthology.orchestra.run.smithi123.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-g2577e813-1focal [3,506 kB] 2024-08-17T02:18:35.405 INFO:teuthology.orchestra.run.smithi123.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-17T02:18:35.407 INFO:teuthology.orchestra.run.smithi123.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-17T02:18:35.487 INFO:teuthology.orchestra.run.smithi123.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-17T02:18:35.488 INFO:teuthology.orchestra.run.smithi123.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-17T02:18:35.489 INFO:teuthology.orchestra.run.smithi123.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-17T02:18:35.490 INFO:teuthology.orchestra.run.smithi123.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-17T02:18:35.767 INFO:teuthology.orchestra.run.smithi123.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-17T02:18:35.842 INFO:teuthology.orchestra.run.smithi123.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-17T02:18:35.846 INFO:teuthology.orchestra.run.smithi123.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-17T02:18:35.914 INFO:teuthology.orchestra.run.smithi123.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-17T02:18:35.914 INFO:teuthology.orchestra.run.smithi123.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-17T02:18:35.915 INFO:teuthology.orchestra.run.smithi123.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-17T02:18:35.916 INFO:teuthology.orchestra.run.smithi123.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-17T02:18:35.916 INFO:teuthology.orchestra.run.smithi123.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-17T02:18:35.916 INFO:teuthology.orchestra.run.smithi123.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-17T02:18:35.916 INFO:teuthology.orchestra.run.smithi123.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-17T02:18:35.972 INFO:teuthology.orchestra.run.smithi123.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-17T02:18:35.973 INFO:teuthology.orchestra.run.smithi123.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-17T02:18:36.054 INFO:teuthology.orchestra.run.smithi123.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-17T02:18:36.058 INFO:teuthology.orchestra.run.smithi123.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-17T02:18:36.061 INFO:teuthology.orchestra.run.smithi123.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-17T02:18:36.063 INFO:teuthology.orchestra.run.smithi123.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-17T02:18:36.063 INFO:teuthology.orchestra.run.smithi123.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-17T02:18:36.067 INFO:teuthology.orchestra.run.smithi123.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-17T02:18:36.068 INFO:teuthology.orchestra.run.smithi123.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-17T02:18:36.068 INFO:teuthology.orchestra.run.smithi123.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-17T02:18:36.128 INFO:teuthology.orchestra.run.smithi100.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-g2577e813-1focal [3,560 kB] 2024-08-17T02:18:36.142 INFO:teuthology.orchestra.run.smithi123.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-17T02:18:36.147 INFO:teuthology.orchestra.run.smithi123.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-17T02:18:36.311 INFO:teuthology.orchestra.run.smithi123.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-17T02:18:36.313 INFO:teuthology.orchestra.run.smithi123.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-17T02:18:36.313 INFO:teuthology.orchestra.run.smithi123.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-17T02:18:36.315 INFO:teuthology.orchestra.run.smithi123.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-17T02:18:36.315 INFO:teuthology.orchestra.run.smithi123.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-17T02:18:36.316 INFO:teuthology.orchestra.run.smithi123.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-17T02:18:36.328 INFO:teuthology.orchestra.run.smithi123.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-17T02:18:36.328 INFO:teuthology.orchestra.run.smithi123.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-17T02:18:36.328 INFO:teuthology.orchestra.run.smithi123.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-17T02:18:36.394 INFO:teuthology.orchestra.run.smithi123.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-17T02:18:36.414 INFO:teuthology.orchestra.run.smithi100.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-g2577e813-1focal [761 kB] 2024-08-17T02:18:36.452 INFO:teuthology.orchestra.run.smithi100.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-g2577e813-1focal [414 kB] 2024-08-17T02:18:36.467 INFO:teuthology.orchestra.run.smithi100.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-g2577e813-1focal [22.6 kB] 2024-08-17T02:18:36.467 INFO:teuthology.orchestra.run.smithi100.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-g2577e813-1focal [200 kB] 2024-08-17T02:18:36.475 INFO:teuthology.orchestra.run.smithi123.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-17T02:18:36.476 INFO:teuthology.orchestra.run.smithi123.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-17T02:18:36.479 INFO:teuthology.orchestra.run.smithi123.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-17T02:18:36.480 INFO:teuthology.orchestra.run.smithi123.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-17T02:18:36.485 INFO:teuthology.orchestra.run.smithi100.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-g2577e813-1focal [49.1 kB] 2024-08-17T02:18:36.486 INFO:teuthology.orchestra.run.smithi100.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-g2577e813-1focal [389 kB] 2024-08-17T02:18:36.513 INFO:teuthology.orchestra.run.smithi100.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-g2577e813-1focal [5,612 kB] 2024-08-17T02:18:36.737 INFO:teuthology.orchestra.run.smithi123.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-g2577e813-1focal [3,560 kB] 2024-08-17T02:18:36.862 INFO:teuthology.orchestra.run.smithi100.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-g2577e813-1focal [116 kB] 2024-08-17T02:18:36.865 INFO:teuthology.orchestra.run.smithi100.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-g2577e813-1focal [478 kB] 2024-08-17T02:18:36.892 INFO:teuthology.orchestra.run.smithi100.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-g2577e813-1focal [26.9 MB] 2024-08-17T02:18:37.340 INFO:teuthology.orchestra.run.smithi123.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-g2577e813-1focal [761 kB] 2024-08-17T02:18:37.463 INFO:teuthology.orchestra.run.smithi123.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-g2577e813-1focal [414 kB] 2024-08-17T02:18:37.529 INFO:teuthology.orchestra.run.smithi123.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-g2577e813-1focal [22.6 kB] 2024-08-17T02:18:37.530 INFO:teuthology.orchestra.run.smithi123.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-g2577e813-1focal [200 kB] 2024-08-17T02:18:37.589 INFO:teuthology.orchestra.run.smithi123.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-g2577e813-1focal [49.1 kB] 2024-08-17T02:18:37.591 INFO:teuthology.orchestra.run.smithi123.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-g2577e813-1focal [389 kB] 2024-08-17T02:18:37.657 INFO:teuthology.orchestra.run.smithi123.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-g2577e813-1focal [5,612 kB] 2024-08-17T02:18:38.214 INFO:teuthology.orchestra.run.smithi100.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-g2577e813-1focal [5,173 kB] 2024-08-17T02:18:38.284 INFO:teuthology.orchestra.run.smithi123.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-g2577e813-1focal [116 kB] 2024-08-17T02:18:38.288 INFO:teuthology.orchestra.run.smithi100.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-g2577e813-1focal [223 kB] 2024-08-17T02:18:38.292 INFO:teuthology.orchestra.run.smithi100.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-g2577e813-1focal [157 kB] 2024-08-17T02:18:38.295 INFO:teuthology.orchestra.run.smithi100.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-g2577e813-1focal [1,594 kB] 2024-08-17T02:18:38.295 INFO:teuthology.orchestra.run.smithi123.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-g2577e813-1focal [478 kB] 2024-08-17T02:18:38.345 INFO:teuthology.orchestra.run.smithi100.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-g2577e813-1focal [4,686 kB] 2024-08-17T02:18:38.348 INFO:teuthology.orchestra.run.smithi123.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-g2577e813-1focal [26.9 MB] 2024-08-17T02:18:38.502 INFO:teuthology.orchestra.run.smithi100.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-g2577e813-1focal [23.1 MB] 2024-08-17T02:18:39.362 INFO:teuthology.orchestra.run.smithi100.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-g2577e813-1focal [3,944 B] 2024-08-17T02:18:39.366 INFO:teuthology.orchestra.run.smithi100.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-g2577e813-1focal [871 kB] 2024-08-17T02:18:39.377 INFO:teuthology.orchestra.run.smithi100.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-g2577e813-1focal [2,415 kB] 2024-08-17T02:18:39.394 INFO:teuthology.orchestra.run.smithi100.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-g2577e813-1focal [81.4 kB] 2024-08-17T02:18:39.394 INFO:teuthology.orchestra.run.smithi100.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-g2577e813-1focal [120 kB] 2024-08-17T02:18:39.395 INFO:teuthology.orchestra.run.smithi100.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-g2577e813-1focal [6,653 kB] 2024-08-17T02:18:39.575 INFO:teuthology.orchestra.run.smithi100.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-g2577e813-1focal [7,886 kB] 2024-08-17T02:18:39.838 INFO:teuthology.orchestra.run.smithi100.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-g2577e813-1focal [4,060 B] 2024-08-17T02:18:39.839 INFO:teuthology.orchestra.run.smithi100.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-g2577e813-1focal [53.3 MB] 2024-08-17T02:18:41.052 INFO:teuthology.orchestra.run.smithi123.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-g2577e813-1focal [5,173 kB] 2024-08-17T02:18:41.368 INFO:teuthology.orchestra.run.smithi123.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-g2577e813-1focal [223 kB] 2024-08-17T02:18:41.374 INFO:teuthology.orchestra.run.smithi123.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-g2577e813-1focal [157 kB] 2024-08-17T02:18:41.385 INFO:teuthology.orchestra.run.smithi123.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-g2577e813-1focal [1,594 kB] 2024-08-17T02:18:41.432 INFO:teuthology.orchestra.run.smithi123.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-g2577e813-1focal [4,686 kB] 2024-08-17T02:18:41.492 INFO:teuthology.orchestra.run.smithi100.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-g2577e813-1focal [230 kB] 2024-08-17T02:18:41.496 INFO:teuthology.orchestra.run.smithi100.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-g2577e813-1focal [241 kB] 2024-08-17T02:18:41.500 INFO:teuthology.orchestra.run.smithi100.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-g2577e813-1focal [22.0 kB] 2024-08-17T02:18:41.500 INFO:teuthology.orchestra.run.smithi100.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-g2577e813-1focal [13.1 kB] 2024-08-17T02:18:41.555 INFO:teuthology.orchestra.run.smithi100.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-g2577e813-1focal [21.9 kB] 2024-08-17T02:18:41.555 INFO:teuthology.orchestra.run.smithi100.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-g2577e813-1focal [17.1 MB] 2024-08-17T02:18:41.580 INFO:teuthology.orchestra.run.smithi123.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-g2577e813-1focal [23.1 MB] 2024-08-17T02:18:41.977 INFO:teuthology.orchestra.run.smithi100.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-g2577e813-1focal [80.6 kB] 2024-08-17T02:18:42.567 INFO:teuthology.orchestra.run.smithi123.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-g2577e813-1focal [3,944 B] 2024-08-17T02:18:42.568 INFO:teuthology.orchestra.run.smithi123.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-g2577e813-1focal [871 kB] 2024-08-17T02:18:42.583 INFO:teuthology.orchestra.run.smithi123.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-g2577e813-1focal [2,415 kB] 2024-08-17T02:18:42.617 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 201 MB in 8s (25.2 MB/s) 2024-08-17T02:18:42.635 INFO:teuthology.orchestra.run.smithi123.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-g2577e813-1focal [81.4 kB] 2024-08-17T02:18:42.651 INFO:teuthology.orchestra.run.smithi123.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-g2577e813-1focal [120 kB] 2024-08-17T02:18:42.651 INFO:teuthology.orchestra.run.smithi123.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-g2577e813-1focal [6,653 kB] 2024-08-17T02:18:42.763 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-17T02:18:42.818 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-17T02:18:42.822 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-17T02:18:42.870 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-17T02:18:42.902 INFO:teuthology.orchestra.run.smithi123.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-g2577e813-1focal [7,886 kB] 2024-08-17T02:18:43.137 INFO:teuthology.orchestra.run.smithi123.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-g2577e813-1focal [4,060 B] 2024-08-17T02:18:43.137 INFO:teuthology.orchestra.run.smithi123.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-g2577e813-1focal [53.3 MB] 2024-08-17T02:18:43.173 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-17T02:18:43.191 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-17T02:18:43.231 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-17T02:18:43.508 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cmd2. 2024-08-17T02:18:43.525 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-17T02:18:43.558 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-17T02:18:43.911 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-17T02:18:43.928 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-17T02:18:44.011 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-17T02:18:44.343 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:44.465 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librbd1 (17.2.7-1375-g2577e813-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-17T02:18:45.031 INFO:teuthology.orchestra.run.smithi123.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-g2577e813-1focal [230 kB] 2024-08-17T02:18:45.036 INFO:teuthology.orchestra.run.smithi123.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-g2577e813-1focal [241 kB] 2024-08-17T02:18:45.041 INFO:teuthology.orchestra.run.smithi123.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-g2577e813-1focal [22.0 kB] 2024-08-17T02:18:45.042 INFO:teuthology.orchestra.run.smithi123.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-g2577e813-1focal [13.1 kB] 2024-08-17T02:18:45.042 INFO:teuthology.orchestra.run.smithi123.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-g2577e813-1focal [21.9 kB] 2024-08-17T02:18:45.044 INFO:teuthology.orchestra.run.smithi123.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-g2577e813-1focal [17.1 MB] 2024-08-17T02:18:45.123 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:45.244 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librados2 (17.2.7-1375-g2577e813-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-17T02:18:45.570 INFO:teuthology.orchestra.run.smithi123.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-g2577e813-1focal [80.6 kB] 2024-08-17T02:18:45.918 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs2. 2024-08-17T02:18:45.935 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:45.967 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:46.208 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 201 MB in 11s (17.6 MB/s) 2024-08-17T02:18:46.269 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rados. 2024-08-17T02:18:46.288 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:46.327 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rados (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:46.352 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-17T02:18:46.407 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-17T02:18:46.412 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-17T02:18:46.458 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-17T02:18:46.680 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-17T02:18:46.699 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:18:46.771 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:46.786 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-17T02:18:46.804 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-17T02:18:46.844 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-17T02:18:47.024 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cephfs. 2024-08-17T02:18:47.040 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:47.082 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cephfs (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:47.172 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cmd2. 2024-08-17T02:18:47.188 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-17T02:18:47.230 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-17T02:18:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-17T02:18:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:18:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-common (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:47.558 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-17T02:18:47.575 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-17T02:18:47.641 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-17T02:18:47.755 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-prettytable. 2024-08-17T02:18:47.770 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-17T02:18:47.815 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-17T02:18:47.962 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:48.057 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rbd. 2024-08-17T02:18:48.062 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librbd1 (17.2.7-1375-g2577e813-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-17T02:18:48.073 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:48.115 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rbd (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:48.502 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-17T02:18:48.524 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-17T02:18:48.560 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-17T02:18:48.757 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:48.855 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-17T02:18:48.875 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-17T02:18:48.904 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-17T02:18:48.927 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librados2 (17.2.7-1375-g2577e813-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-17T02:18:49.258 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-17T02:18:49.276 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-17T02:18:49.316 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-17T02:18:49.594 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libcephfs2. 2024-08-17T02:18:49.616 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:49.644 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libcephfs2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:49.719 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-17T02:18:49.738 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-17T02:18:49.777 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-17T02:18:49.955 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-rados. 2024-08-17T02:18:49.973 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:50.006 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-rados (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:50.063 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-17T02:18:50.079 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-17T02:18:50.113 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-17T02:18:50.386 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-17T02:18:50.402 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:18:50.443 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:50.542 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-17T02:18:50.561 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-17T02:18:50.602 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:18:50.695 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cephfs. 2024-08-17T02:18:50.711 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:50.745 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cephfs (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:51.015 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-17T02:18:51.031 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:18:51.064 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-ceph-common (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:51.086 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-17T02:18:51.102 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-17T02:18:51.144 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:18:51.434 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-prettytable. 2024-08-17T02:18:51.451 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-17T02:18:51.493 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-17T02:18:51.506 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-17T02:18:51.524 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-17T02:18:51.623 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:18:51.778 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-rbd. 2024-08-17T02:18:51.795 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:51.878 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-rbd (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:52.093 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-17T02:18:52.111 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-17T02:18:52.142 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-17T02:18:52.356 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-17T02:18:52.373 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-17T02:18:52.416 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-17T02:18:52.479 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librgw2. 2024-08-17T02:18:52.494 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:52.537 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librgw2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:52.792 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-17T02:18:52.809 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-17T02:18:52.850 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-17T02:18:53.212 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-17T02:18:53.227 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-17T02:18:53.270 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-17T02:18:53.331 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rgw. 2024-08-17T02:18:53.347 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:53.415 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rgw (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-17T02:18:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-17T02:18:53.706 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-17T02:18:53.750 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liboath0. 2024-08-17T02:18:53.768 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-17T02:18:53.809 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-17T02:18:54.059 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-17T02:18:54.070 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libradosstriper1. 2024-08-17T02:18:54.077 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-17T02:18:54.087 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:54.117 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-17T02:18:54.153 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libradosstriper1 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:54.604 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-17T02:18:54.606 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-common. 2024-08-17T02:18:54.620 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-17T02:18:54.624 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:54.664 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-common (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:54.672 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:18:55.207 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-17T02:18:55.223 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-17T02:18:55.265 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:18:55.644 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-17T02:18:55.661 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-17T02:18:55.710 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:18:56.112 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-17T02:18:56.132 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-17T02:18:56.170 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-17T02:18:56.540 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package librgw2. 2024-08-17T02:18:56.559 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:56.598 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librgw2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:57.220 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-base. 2024-08-17T02:18:57.236 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:57.289 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-base (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:57.392 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-rgw. 2024-08-17T02:18:57.409 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:57.451 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-rgw (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:57.829 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liboath0. 2024-08-17T02:18:57.846 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-17T02:18:57.887 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-17T02:18:58.158 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cryptography. 2024-08-17T02:18:58.176 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-17T02:18:58.216 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-17T02:18:58.227 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libradosstriper1. 2024-08-17T02:18:58.239 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:58.281 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libradosstriper1 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:58.628 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-openssl. 2024-08-17T02:18:58.644 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-17T02:18:58.686 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-17T02:18:58.776 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-common. 2024-08-17T02:18:58.792 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:18:58.834 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-common (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:58.989 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-packaging. 2024-08-17T02:18:59.005 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-17T02:18:59.048 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-17T02:18:59.300 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-17T02:18:59.319 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:18:59.375 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:18:59.747 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-17T02:18:59.762 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-17T02:18:59.796 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-17T02:19:00.133 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-17T02:19:00.149 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-17T02:19:00.190 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-17T02:19:00.527 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-natsort. 2024-08-17T02:19:00.544 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-17T02:19:00.585 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-17T02:19:00.939 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jwt. 2024-08-17T02:19:00.955 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-17T02:19:01.005 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-17T02:19:01.297 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-base. 2024-08-17T02:19:01.316 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:01.317 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-logutils. 2024-08-17T02:19:01.333 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-17T02:19:01.366 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-base (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:01.375 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-17T02:19:01.686 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-17T02:19:01.702 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-17T02:19:01.744 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-17T02:19:02.097 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-mako. 2024-08-17T02:19:02.117 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-17T02:19:02.157 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-17T02:19:02.277 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cryptography. 2024-08-17T02:19:02.296 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-17T02:19:02.352 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-17T02:19:02.501 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-17T02:19:02.517 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-17T02:19:02.558 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-17T02:19:02.763 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-openssl. 2024-08-17T02:19:02.779 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-17T02:19:02.820 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-17T02:19:02.821 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-17T02:19:02.836 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-17T02:19:02.869 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-17T02:19:03.175 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-packaging. 2024-08-17T02:19:03.189 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webob. 2024-08-17T02:19:03.190 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-17T02:19:03.205 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-17T02:19:03.233 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-17T02:19:03.247 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-17T02:19:03.510 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-17T02:19:03.526 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:03.568 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:03.575 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-17T02:19:03.595 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-17T02:19:03.633 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-17T02:19:03.961 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-bs4. 2024-08-17T02:19:03.973 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-17T02:19:03.977 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-17T02:19:03.990 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-17T02:19:04.019 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-17T02:19:04.031 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-17T02:19:04.359 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-17T02:19:04.377 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-17T02:19:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-waitress. 2024-08-17T02:19:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-17T02:19:04.417 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-17T02:19:04.434 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-17T02:19:04.721 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-natsort. 2024-08-17T02:19:04.740 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-17T02:19:04.784 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempita. 2024-08-17T02:19:04.800 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-17T02:19:04.804 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-17T02:19:04.842 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-17T02:19:05.115 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-jwt. 2024-08-17T02:19:05.131 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-17T02:19:05.145 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-paste. 2024-08-17T02:19:05.161 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-17T02:19:05.173 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-17T02:19:05.226 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-17T02:19:05.493 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-logutils. 2024-08-17T02:19:05.512 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-17T02:19:05.548 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-17T02:19:05.560 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-17T02:19:05.564 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-17T02:19:05.606 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-17T02:19:05.862 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-17T02:19:05.879 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-17T02:19:05.915 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-17T02:19:05.920 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-17T02:19:05.931 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-17T02:19:06.006 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-17T02:19:06.173 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-mako. 2024-08-17T02:19:06.193 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-17T02:19:06.231 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-17T02:19:06.310 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webtest. 2024-08-17T02:19:06.326 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-17T02:19:06.368 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-17T02:19:06.534 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-17T02:19:06.551 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-17T02:19:06.625 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-17T02:19:06.705 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pecan. 2024-08-17T02:19:06.723 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-17T02:19:06.763 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-17T02:19:06.928 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-17T02:19:06.946 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-17T02:19:06.987 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-17T02:19:07.118 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-17T02:19:07.136 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-17T02:19:07.323 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-webob. 2024-08-17T02:19:07.342 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-17T02:19:07.381 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-17T02:19:07.451 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-17T02:19:08.238 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-17T02:19:08.256 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:08.304 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:08.509 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-17T02:19:08.528 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-17T02:19:08.575 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-17T02:19:08.658 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr. 2024-08-17T02:19:08.674 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:08.716 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:08.820 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-bs4. 2024-08-17T02:19:08.837 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-17T02:19:08.878 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-17T02:19:09.206 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-waitress. 2024-08-17T02:19:09.225 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-17T02:19:09.261 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mon. 2024-08-17T02:19:09.267 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-17T02:19:09.282 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:09.311 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mon (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:09.608 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-tempita. 2024-08-17T02:19:09.628 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-17T02:19:09.666 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-17T02:19:09.995 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-paste. 2024-08-17T02:19:10.010 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-17T02:19:10.052 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-17T02:19:10.126 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-osd. 2024-08-17T02:19:10.142 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:10.184 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-osd (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:10.402 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-17T02:19:10.419 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-17T02:19:10.469 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-17T02:19:10.797 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-17T02:19:10.816 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-17T02:19:10.855 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-17T02:19:11.158 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-webtest. 2024-08-17T02:19:11.175 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-17T02:19:11.241 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-17T02:19:11.544 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pecan. 2024-08-17T02:19:11.560 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-17T02:19:11.602 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-17T02:19:11.956 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-17T02:19:11.976 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-17T02:19:12.014 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-17T02:19:12.362 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph. 2024-08-17T02:19:12.376 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-17T02:19:12.380 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:12.394 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:12.412 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:12.757 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-fuse. 2024-08-17T02:19:12.774 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:12.796 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr. 2024-08-17T02:19:12.814 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:12.815 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-fuse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:12.854 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:13.301 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mds. 2024-08-17T02:19:13.317 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:13.358 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mds (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:13.377 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mon. 2024-08-17T02:19:13.394 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:13.444 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mon (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:13.887 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephadm. 2024-08-17T02:19:13.903 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:13.944 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephadm (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:14.248 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-osd. 2024-08-17T02:19:14.264 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:14.264 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jinja2. 2024-08-17T02:19:14.283 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-17T02:19:14.305 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-osd (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-17T02:19:14.601 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-17T02:19:14.617 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:14.651 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:15.012 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-17T02:19:15.031 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-17T02:19:15.070 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-17T02:19:15.323 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjs-underscore. 2024-08-17T02:19:15.340 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-17T02:19:15.381 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-17T02:19:15.709 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-17T02:19:15.725 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-17T02:19:15.767 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-17T02:19:16.053 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-routes. 2024-08-17T02:19:16.069 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-17T02:19:16.102 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-17T02:19:16.357 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph. 2024-08-17T02:19:16.376 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:16.415 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:16.439 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-17T02:19:16.456 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:16.497 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:16.759 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-fuse. 2024-08-17T02:19:16.780 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:16.817 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-fuse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:17.263 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mds. 2024-08-17T02:19:17.281 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:17.312 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mds (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:17.819 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-decorator. 2024-08-17T02:19:17.839 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-17T02:19:17.856 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package cephadm. 2024-08-17T02:19:17.874 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:17.896 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-17T02:19:17.914 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking cephadm (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:18.189 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-17T02:19:18.205 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-17T02:19:18.247 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-17T02:19:18.268 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-jinja2. 2024-08-17T02:19:18.284 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-17T02:19:18.326 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-17T02:19:18.575 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-scipy. 2024-08-17T02:19:18.590 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-17T02:19:18.657 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-17T02:19:18.704 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-17T02:19:18.722 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:18.762 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:19.140 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-17T02:19:19.158 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-17T02:19:19.198 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-17T02:19:19.476 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libjs-underscore. 2024-08-17T02:19:19.497 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-17T02:19:19.534 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-17T02:19:19.803 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-17T02:19:19.821 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-17T02:19:19.861 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-17T02:19:20.000 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-17T02:19:20.019 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-17T02:19:20.058 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-17T02:19:20.214 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-routes. 2024-08-17T02:19:20.230 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-17T02:19:20.272 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-17T02:19:20.571 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-joblib. 2024-08-17T02:19:20.575 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-17T02:19:20.591 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:20.591 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-17T02:19:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:20.629 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-17T02:19:21.001 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn. 2024-08-17T02:19:21.020 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-17T02:19:21.059 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-17T02:19:21.718 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-17T02:19:21.734 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:21.767 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:21.955 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-decorator. 2024-08-17T02:19:21.971 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-17T02:19:22.015 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-17T02:19:22.332 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-17T02:19:22.348 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-17T02:19:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-17T02:19:22.726 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-scipy. 2024-08-17T02:19:22.742 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-17T02:19:22.784 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-17T02:19:22.896 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cachetools. 2024-08-17T02:19:22.911 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-17T02:19:22.953 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-17T02:19:23.303 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-17T02:19:23.323 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-17T02:19:23.682 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-17T02:19:24.043 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-17T02:19:24.061 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-17T02:19:24.101 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-17T02:19:24.379 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rsa. 2024-08-17T02:19:24.386 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-17T02:19:24.399 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-17T02:19:24.403 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-17T02:19:24.437 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-17T02:19:24.453 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-17T02:19:24.774 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-google-auth. 2024-08-17T02:19:24.790 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-17T02:19:24.832 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-17T02:19:24.990 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-joblib. 2024-08-17T02:19:25.006 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-17T02:19:25.048 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-17T02:19:25.135 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-blinker. 2024-08-17T02:19:25.155 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-17T02:19:25.193 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-17T02:19:25.444 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-sklearn. 2024-08-17T02:19:25.461 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-17T02:19:25.501 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-17T02:19:25.538 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-17T02:19:25.555 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-17T02:19:25.588 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-17T02:19:26.018 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-17T02:19:26.034 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-17T02:19:26.076 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-17T02:19:26.192 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-17T02:19:26.210 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:26.242 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:26.371 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-websocket. 2024-08-17T02:19:26.391 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-17T02:19:26.431 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-17T02:19:26.759 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-17T02:19:26.779 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-17T02:19:26.817 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-17T02:19:27.339 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-17T02:19:27.359 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:27.389 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:27.397 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cachetools. 2024-08-17T02:19:27.414 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-17T02:19:27.455 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-17T02:19:27.742 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-17T02:19:27.758 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-17T02:19:27.774 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-17T02:19:27.793 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-17T02:19:27.809 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-17T02:19:27.840 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-17T02:19:28.162 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-17T02:19:28.180 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-17T02:19:28.186 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-17T02:19:28.205 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-17T02:19:28.237 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-17T02:19:28.243 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-17T02:19:28.465 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package jq. 2024-08-17T02:19:28.482 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-17T02:19:28.514 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-17T02:19:28.555 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-rsa. 2024-08-17T02:19:28.570 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-17T02:19:28.638 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-17T02:19:28.771 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package socat. 2024-08-17T02:19:28.789 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-17T02:19:28.829 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-17T02:19:28.924 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-google-auth. 2024-08-17T02:19:28.944 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-17T02:19:28.982 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-17T02:19:29.157 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package xmlstarlet. 2024-08-17T02:19:29.173 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-17T02:19:29.215 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-17T02:19:29.327 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-blinker. 2024-08-17T02:19:29.345 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-17T02:19:29.385 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-17T02:19:29.537 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-test. 2024-08-17T02:19:29.553 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:29.594 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-test (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:29.712 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-17T02:19:29.730 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-17T02:19:29.770 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-17T02:19:30.235 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-17T02:19:30.251 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-17T02:19:30.293 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-17T02:19:30.605 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-websocket. 2024-08-17T02:19:30.624 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-17T02:19:30.671 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-17T02:19:31.008 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-17T02:19:31.026 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-17T02:19:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-17T02:19:31.623 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-17T02:19:31.640 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:31.680 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:32.025 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-17T02:19:32.042 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-17T02:19:32.084 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-17T02:19:32.461 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-17T02:19:32.478 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-17T02:19:32.519 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-17T02:19:32.809 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package jq. 2024-08-17T02:19:32.826 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-17T02:19:32.867 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-17T02:19:33.103 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package socat. 2024-08-17T02:19:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-17T02:19:33.181 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-17T02:19:33.506 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package xmlstarlet. 2024-08-17T02:19:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-17T02:19:33.572 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-17T02:19:33.918 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-test. 2024-08-17T02:19:33.934 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:33.976 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-test (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:34.165 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-volume. 2024-08-17T02:19:34.182 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:34.223 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-volume (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:34.646 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-17T02:19:34.664 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:34.704 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-mirror (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:35.074 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-colorama. 2024-08-17T02:19:35.090 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-17T02:19:35.127 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-17T02:19:35.388 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-shell. 2024-08-17T02:19:35.405 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:35.446 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-shell (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:35.733 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-top. 2024-08-17T02:19:35.750 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:35.791 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-top (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:36.029 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-17T02:19:36.048 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-17T02:19:36.093 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-17T02:19:36.608 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-17T02:19:36.624 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:36.665 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs-dev (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:37.086 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package nvme-cli. 2024-08-17T02:19:37.104 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-17T02:19:37.161 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-17T02:19:37.490 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-17T02:19:37.509 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-17T02:19:37.548 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-17T02:19:38.414 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-volume. 2024-08-17T02:19:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-volume (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-17T02:19:38.894 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:38.946 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking cephfs-mirror (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:39.152 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-17T02:19:39.173 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-17T02:19:39.209 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-17T02:19:39.316 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-colorama. 2024-08-17T02:19:39.334 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-17T02:19:39.374 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-17T02:19:39.563 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-17T02:19:39.581 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-17T02:19:39.621 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-17T02:19:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package cephfs-shell. 2024-08-17T02:19:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking cephfs-shell (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:39.907 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-attr. 2024-08-17T02:19:39.928 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-17T02:19:39.965 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-17T02:19:39.979 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package cephfs-top. 2024-08-17T02:19:39.995 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-g2577e813-1focal_all.deb ... 2024-08-17T02:19:40.037 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking cephfs-top (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:40.317 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-17T02:19:40.333 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-17T02:19:40.334 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webencodings. 2024-08-17T02:19:40.351 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-17T02:19:40.375 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-17T02:19:40.392 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-17T02:19:40.662 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-html5lib. 2024-08-17T02:19:40.681 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-17T02:19:40.711 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-17T02:19:40.920 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-17T02:19:40.936 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:40.978 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libcephfs-dev (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:41.023 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-17T02:19:41.044 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-17T02:19:41.081 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-17T02:19:41.423 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package nvme-cli. 2024-08-17T02:19:41.441 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-17T02:19:41.498 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-17T02:19:41.560 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastescript. 2024-08-17T02:19:41.577 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-17T02:19:41.618 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-17T02:19:41.852 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-17T02:19:41.872 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-17T02:19:41.910 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-17T02:19:41.946 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pluggy. 2024-08-17T02:19:41.965 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-17T02:19:42.004 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-17T02:19:42.315 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-psutil. 2024-08-17T02:19:42.336 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-17T02:19:42.375 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-17T02:19:42.710 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-py. 2024-08-17T02:19:42.729 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-17T02:19:42.768 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-17T02:19:43.121 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-17T02:19:43.138 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-17T02:19:43.179 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-17T02:19:43.490 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pytest. 2024-08-17T02:19:43.507 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-17T02:19:43.548 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-17T02:19:43.585 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-17T02:19:43.604 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-17T02:19:43.643 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-17T02:19:43.919 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplejson. 2024-08-17T02:19:43.935 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-17T02:19:43.976 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-17T02:19:43.996 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-17T02:19:44.013 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-17T02:19:44.054 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-17T02:19:44.263 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-17T02:19:44.279 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-17T02:19:44.321 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-17T02:19:44.340 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-attr. 2024-08-17T02:19:44.361 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-17T02:19:44.431 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-17T02:19:44.776 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-webencodings. 2024-08-17T02:19:44.794 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-17T02:19:44.867 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-17T02:19:45.109 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package radosgw. 2024-08-17T02:19:45.125 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:45.170 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-html5lib. 2024-08-17T02:19:45.187 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-17T02:19:45.208 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking radosgw (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:45.229 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-17T02:19:45.574 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-17T02:19:45.593 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-17T02:19:45.631 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-17T02:19:46.109 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pastescript. 2024-08-17T02:19:46.127 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-17T02:19:46.167 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-17T02:19:46.512 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pluggy. 2024-08-17T02:19:46.530 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-17T02:19:46.570 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-17T02:19:46.915 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-psutil. 2024-08-17T02:19:46.932 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-17T02:19:46.973 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-17T02:19:46.990 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package rbd-fuse. 2024-08-17T02:19:47.010 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:47.048 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking rbd-fuse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:47.334 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-py. 2024-08-17T02:19:47.351 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-17T02:19:47.353 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-17T02:19:47.392 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-17T02:19:47.644 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-17T02:19:47.729 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-17T02:19:47.747 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-17T02:19:47.772 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-17T02:19:47.787 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-17T02:19:47.898 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-17T02:19:48.066 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephadm (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:48.098 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pytest. 2024-08-17T02:19:48.117 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-17T02:19:48.155 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-17T02:19:48.476 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-simplejson. 2024-08-17T02:19:48.497 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-17T02:19:48.567 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-17T02:19:48.594 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user cephadm....done 2024-08-17T02:19:48.774 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-17T02:19:48.845 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-17T02:19:48.865 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-17T02:19:48.876 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-17T02:19:48.903 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-17T02:19:49.244 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-17T02:19:49.529 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-17T02:19:49.679 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-17T02:19:49.709 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package radosgw. 2024-08-17T02:19:49.724 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:49.768 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking radosgw (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:49.814 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-17T02:19:50.149 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-17T02:19:50.517 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-17T02:19:50.793 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-17T02:19:51.088 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-17T02:19:51.414 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-17T02:19:51.513 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package rbd-fuse. 2024-08-17T02:19:51.530 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-g2577e813-1focal_amd64.deb ... 2024-08-17T02:19:51.571 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking rbd-fuse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:51.698 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-17T02:19:51.899 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-17T02:19:51.991 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-argparse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:52.218 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-17T02:19:52.267 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-17T02:19:52.343 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-17T02:19:52.376 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-17T02:19:52.469 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-17T02:19:52.661 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-17T02:19:52.696 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up cephadm (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:52.937 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-17T02:19:53.205 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-17T02:19:53.284 INFO:teuthology.orchestra.run.smithi123.stdout:Adding system user cephadm....done 2024-08-17T02:19:53.364 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-17T02:19:53.420 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-17T02:19:53.547 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-17T02:19:53.732 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-17T02:19:53.939 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-17T02:19:54.000 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-17T02:19:54.257 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-17T02:19:54.368 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-17T02:19:54.403 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-17T02:19:54.527 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-17T02:19:55.177 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-17T02:19:55.196 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-17T02:19:55.461 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-17T02:19:55.591 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-17T02:19:55.779 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-17T02:19:55.906 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-17T02:19:56.073 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-17T02:19:56.218 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-17T02:19:56.365 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-17T02:19:56.474 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-17T02:19:56.494 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-17T02:19:56.575 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-17T02:19:56.770 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-17T02:19:56.843 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-17T02:19:57.063 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-ceph-argparse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:57.228 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-17T02:19:57.364 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-17T02:19:57.490 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-17T02:19:57.513 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-17T02:19:57.783 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-17T02:19:57.873 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-17T02:19:58.067 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-17T02:19:58.233 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-17T02:19:58.334 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-common (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:19:58.385 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-17T02:19:58.486 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-17T02:19:58.627 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:19:58.761 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-17T02:19:58.845 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-17T02:19:59.121 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-17T02:19:59.139 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-17T02:19:59.264 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-17T02:19:59.398 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-17T02:19:59.431 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-17T02:19:59.691 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-17T02:19:59.791 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-17T02:19:59.975 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:20:00.092 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-17T02:20:00.119 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-17T02:20:00.244 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-17T02:20:00.468 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-17T02:20:00.554 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-17T02:20:00.752 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-17T02:20:00.822 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-17T02:20:01.020 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-17T02:20:01.089 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-17T02:20:01.121 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-17T02:20:01.215 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-17T02:20:01.247 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-17T02:20:01.525 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-17T02:20:01.565 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-17T02:20:01.801 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-17T02:20:01.966 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-17T02:20:02.103 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-17T02:20:02.276 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-17T02:20:02.461 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-17T02:20:02.644 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-17T02:20:02.738 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-17T02:20:02.864 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-17T02:20:03.004 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-17T02:20:03.147 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-ceph-common (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:03.215 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-17T02:20:03.465 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:20:03.566 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-17T02:20:03.616 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-17T02:20:03.719 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-17T02:20:03.843 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-17T02:20:03.959 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-17T02:20:04.102 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-17T02:20:04.186 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-17T02:20:04.219 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-17T02:20:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librados2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-17T02:20:04.605 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-17T02:20:04.821 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:20:04.981 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-17T02:20:05.107 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-17T02:20:05.450 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-17T02:20:05.751 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-17T02:20:06.036 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-17T02:20:06.060 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:06.137 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-17T02:20:06.186 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-17T02:20:06.312 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-17T02:20:06.429 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-17T02:20:06.689 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-17T02:20:06.722 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-17T02:20:06.973 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-17T02:20:07.040 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-17T02:20:07.266 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:20:07.412 INFO:teuthology.orchestra.run.smithi123.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-17T02:20:07.417 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-17T02:20:07.701 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-17T02:20:07.768 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:07.802 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-17T02:20:07.894 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-17T02:20:08.128 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-17T02:20:08.221 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libradosstriper1 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:08.338 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-17T02:20:08.505 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-17T02:20:08.656 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-17T02:20:08.665 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-17T02:20:08.783 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-17T02:20:09.116 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-17T02:20:09.426 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librados2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:09.577 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-17T02:20:09.720 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librbd1 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:09.846 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-17T02:20:09.980 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-17T02:20:10.115 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-17T02:20:10.265 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-17T02:20:10.608 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-17T02:20:10.734 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-17T02:20:11.015 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:11.112 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:11.141 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-17T02:20:11.230 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-fuse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:11.277 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-17T02:20:11.456 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-17T02:20:11.456 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-17T02:20:11.677 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-17T02:20:11.882 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs-dev (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:11.954 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-17T02:20:12.008 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-17T02:20:12.288 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-17T02:20:12.301 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-17T02:20:12.414 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-17T02:20:12.619 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-17T02:20:12.774 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libcephfs2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:12.899 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-17T02:20:12.980 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rados (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:13.106 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librgw2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:13.243 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libradosstriper1 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:13.251 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-top (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:13.344 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-17T02:20:13.374 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rbd (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:13.525 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up rbd-fuse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:13.651 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-17T02:20:13.679 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-17T02:20:13.988 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-17T02:20:14.304 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rgw (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:14.421 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cephfs (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-shell (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:14.674 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-17T02:20:14.719 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librbd1 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:14.843 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-17T02:20:14.944 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-17T02:20:15.079 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-17T02:20:15.205 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-17T02:20:15.506 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-17T02:20:15.627 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-common (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:15.632 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-17T02:20:16.035 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:16.136 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-fuse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:16.163 INFO:teuthology.orchestra.run.smithi100.stdout:Adding group ceph....done 2024-08-17T02:20:16.348 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-17T02:20:16.348 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-17T02:20:16.523 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user ceph....done 2024-08-17T02:20:16.700 INFO:teuthology.orchestra.run.smithi100.stdout:Setting system user ceph properties....done 2024-08-17T02:20:16.726 INFO:teuthology.orchestra.run.smithi100.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-17T02:20:16.829 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libcephfs-dev (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:16.873 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-17T02:20:16.963 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-17T02:20:17.200 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-17T02:20:17.290 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-17T02:20:17.608 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-17T02:20:17.677 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-mirror (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:17.908 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-17T02:20:17.908 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-17T02:20:17.960 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-rados (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:18.070 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librgw2 (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:18.221 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up cephfs-top (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:18.339 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-test (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:18.346 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-rbd (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:18.473 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up rbd-fuse (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:18.481 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up radosgw (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:18.641 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-17T02:20:18.753 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-17T02:20:18.754 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-17T02:20:18.941 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-17T02:20:19.251 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-rgw (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:19.353 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cephfs (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-base (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:19.462 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up cephfs-shell (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:19.621 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-17T02:20:19.755 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-17T02:20:20.202 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mds (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:20.429 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-17T02:20:20.429 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-17T02:20:20.566 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-common (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:20.862 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:21.092 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-17T02:20:21.092 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-17T02:20:21.135 INFO:teuthology.orchestra.run.smithi123.stdout:Adding group ceph....done 2024-08-17T02:20:21.537 INFO:teuthology.orchestra.run.smithi123.stdout:Adding system user ceph....done 2024-08-17T02:20:21.555 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-osd (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:21.713 INFO:teuthology.orchestra.run.smithi123.stdout:Setting system user ceph properties....done 2024-08-17T02:20:21.739 INFO:teuthology.orchestra.run.smithi123.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-17T02:20:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-17T02:20:22.042 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-17T02:20:22.042 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-17T02:20:22.214 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-17T02:20:22.499 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:22.624 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:22.691 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up cephfs-mirror (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:22.751 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mon (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:22.917 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-17T02:20:22.917 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-17T02:20:22.995 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-17T02:20:22.995 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-17T02:20:23.360 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-test (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:23.452 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:23.486 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up radosgw (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:23.620 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:23.746 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:23.791 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-17T02:20:23.791 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-17T02:20:23.889 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-volume (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:24.016 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-17T02:20:24.167 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-17T02:20:24.438 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-base (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:24.452 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-17T02:20:24.830 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-17T02:20:25.264 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mds (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:25.518 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-17T02:20:25.518 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-17T02:20:26.152 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-17T02:20:26.422 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-17T02:20:26.868 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-osd (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:27.337 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-17T02:20:27.351 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-17T02:20:27.351 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-17T02:20:27.778 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:27.904 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:27.940 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-17T02:20:27.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-17T02:20:28.015 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-08-17T02:20:28.038 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mon (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:28.231 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-08-17T02:20:28.231 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-08-17T02:20:28.291 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-17T02:20:28.291 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-17T02:20:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-17T02:20:28.451 INFO:teuthology.orchestra.run.smithi100.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-17T02:20:28.451 INFO:teuthology.orchestra.run.smithi100.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-17T02:20:28.451 INFO:teuthology.orchestra.run.smithi100.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-17T02:20:28.451 INFO:teuthology.orchestra.run.smithi100.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-17T02:20:28.451 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-08-17T02:20:28.452 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-17T02:20:28.452 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-17T02:20:28.452 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-08-17T02:20:28.452 INFO:teuthology.orchestra.run.smithi100.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-17T02:20:28.452 INFO:teuthology.orchestra.run.smithi100.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-17T02:20:28.453 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-17T02:20:28.453 INFO:teuthology.orchestra.run.smithi100.stdout: usb-modeswitch-data wpasupplicant 2024-08-17T02:20:28.454 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-08-17T02:20:28.454 INFO:teuthology.orchestra.run.smithi100.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-17T02:20:28.454 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-17T02:20:28.507 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-08-17T02:20:28.507 INFO:teuthology.orchestra.run.smithi100.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-17T02:20:28.507 INFO:teuthology.orchestra.run.smithi100.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-17T02:20:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-17T02:20:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-17T02:20:28.509 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-08-17T02:20:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-17T02:20:28.566 INFO:teuthology.orchestra.run.smithi100.stdout:6 upgraded, 23 newly installed, 0 to remove and 289 not upgraded. 2024-08-17T02:20:28.566 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 10.4 MB of archives. 2024-08-17T02:20:28.566 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-17T02:20:28.566 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-17T02:20:28.699 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-17T02:20:28.700 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-17T02:20:28.701 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-17T02:20:28.706 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-17T02:20:28.708 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-17T02:20:28.724 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-17T02:20:28.724 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-17T02:20:28.726 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-17T02:20:28.727 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-17T02:20:28.727 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-17T02:20:28.733 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-17T02:20:28.734 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-17T02:20:28.735 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-17T02:20:28.735 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-17T02:20:28.739 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:28.750 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-17T02:20:28.751 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-17T02:20:28.757 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-17T02:20:28.765 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-17T02:20:28.765 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-17T02:20:28.782 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-17T02:20:28.782 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-17T02:20:28.787 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-17T02:20:28.803 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-17T02:20:28.814 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-17T02:20:28.815 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-17T02:20:28.816 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-17T02:20:28.816 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-17T02:20:28.817 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-17T02:20:28.882 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:29.041 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:29.129 INFO:teuthology.orchestra.run.smithi100.stdout:Preconfiguring packages ... 2024-08-17T02:20:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-volume (17.2.7-1375-g2577e813-1focal) ... 2024-08-17T02:20:29.247 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 10.4 MB in 0s (34.9 MB/s) 2024-08-17T02:20:29.294 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-17T02:20:29.446 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-17T02:20:29.546 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-17T02:20:29.551 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-17T02:20:29.690 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-17T02:20:29.755 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-17T02:20:30.782 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-17T02:20:32.567 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-17T02:20:32.568 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-17T02:20:32.571 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-17T02:20:32.696 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-17T02:20:33.094 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-17T02:20:33.186 INFO:teuthology.orchestra.run.smithi123.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-17T02:20:33.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-17T02:20:33.261 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2024-08-17T02:20:33.277 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-17T02:20:33.478 INFO:teuthology.orchestra.run.smithi123.stdout:Building dependency tree... 2024-08-17T02:20:33.480 INFO:teuthology.orchestra.run.smithi123.stdout:Reading state information... 2024-08-17T02:20:33.621 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-17T02:20:33.698 INFO:teuthology.orchestra.run.smithi123.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-17T02:20:33.698 INFO:teuthology.orchestra.run.smithi123.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-17T02:20:33.698 INFO:teuthology.orchestra.run.smithi123.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-17T02:20:33.698 INFO:teuthology.orchestra.run.smithi123.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-17T02:20:33.698 INFO:teuthology.orchestra.run.smithi123.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-17T02:20:33.698 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages were automatically installed and are no longer required: 2024-08-17T02:20:33.699 INFO:teuthology.orchestra.run.smithi123.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-17T02:20:33.699 INFO:teuthology.orchestra.run.smithi123.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-17T02:20:33.700 INFO:teuthology.orchestra.run.smithi123.stdout:The following additional packages will be installed: 2024-08-17T02:20:33.700 INFO:teuthology.orchestra.run.smithi123.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-17T02:20:33.700 INFO:teuthology.orchestra.run.smithi123.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-17T02:20:33.700 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-17T02:20:33.700 INFO:teuthology.orchestra.run.smithi123.stdout: usb-modeswitch-data wpasupplicant 2024-08-17T02:20:33.702 INFO:teuthology.orchestra.run.smithi123.stdout:Suggested packages: 2024-08-17T02:20:33.702 INFO:teuthology.orchestra.run.smithi123.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-17T02:20:33.702 INFO:teuthology.orchestra.run.smithi123.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-17T02:20:33.753 INFO:teuthology.orchestra.run.smithi123.stdout:The following NEW packages will be installed: 2024-08-17T02:20:33.753 INFO:teuthology.orchestra.run.smithi123.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-17T02:20:33.753 INFO:teuthology.orchestra.run.smithi123.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-17T02:20:33.753 INFO:teuthology.orchestra.run.smithi123.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-17T02:20:33.754 INFO:teuthology.orchestra.run.smithi123.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-17T02:20:33.754 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages will be upgraded: 2024-08-17T02:20:33.755 INFO:teuthology.orchestra.run.smithi123.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-17T02:20:33.810 INFO:teuthology.orchestra.run.smithi123.stdout:6 upgraded, 23 newly installed, 0 to remove and 289 not upgraded. 2024-08-17T02:20:33.811 INFO:teuthology.orchestra.run.smithi123.stdout:Need to get 10.4 MB of archives. 2024-08-17T02:20:33.811 INFO:teuthology.orchestra.run.smithi123.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-17T02:20:33.811 INFO:teuthology.orchestra.run.smithi123.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-17T02:20:33.942 INFO:teuthology.orchestra.run.smithi123.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-17T02:20:33.942 INFO:teuthology.orchestra.run.smithi123.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-17T02:20:33.943 INFO:teuthology.orchestra.run.smithi123.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-17T02:20:33.948 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-17T02:20:33.949 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-17T02:20:33.964 INFO:teuthology.orchestra.run.smithi123.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-17T02:20:33.965 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-17T02:20:33.966 INFO:teuthology.orchestra.run.smithi123.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-17T02:20:33.967 INFO:teuthology.orchestra.run.smithi123.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-17T02:20:33.967 INFO:teuthology.orchestra.run.smithi123.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-17T02:20:33.973 INFO:teuthology.orchestra.run.smithi123.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-17T02:20:33.974 INFO:teuthology.orchestra.run.smithi123.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-17T02:20:33.975 INFO:teuthology.orchestra.run.smithi123.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-17T02:20:33.976 INFO:teuthology.orchestra.run.smithi123.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-17T02:20:33.990 INFO:teuthology.orchestra.run.smithi123.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-17T02:20:33.991 INFO:teuthology.orchestra.run.smithi123.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-17T02:20:33.996 INFO:teuthology.orchestra.run.smithi123.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-17T02:20:34.004 INFO:teuthology.orchestra.run.smithi123.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-17T02:20:34.004 INFO:teuthology.orchestra.run.smithi123.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-17T02:20:34.018 INFO:teuthology.orchestra.run.smithi123.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-17T02:20:34.018 INFO:teuthology.orchestra.run.smithi123.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-17T02:20:34.028 INFO:teuthology.orchestra.run.smithi123.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-17T02:20:34.036 INFO:teuthology.orchestra.run.smithi123.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-17T02:20:34.052 INFO:teuthology.orchestra.run.smithi123.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-17T02:20:34.054 INFO:teuthology.orchestra.run.smithi123.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-17T02:20:34.055 INFO:teuthology.orchestra.run.smithi123.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-17T02:20:34.055 INFO:teuthology.orchestra.run.smithi123.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-17T02:20:34.055 INFO:teuthology.orchestra.run.smithi123.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-17T02:20:34.066 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-17T02:20:34.070 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-17T02:20:34.185 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-17T02:20:34.372 INFO:teuthology.orchestra.run.smithi123.stdout:Preconfiguring packages ... 2024-08-17T02:20:34.527 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 10.4 MB in 0s (35.7 MB/s) 2024-08-17T02:20:34.674 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-17T02:20:34.807 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-17T02:20:34.808 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-17T02:20:34.813 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-17T02:20:35.001 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-17T02:20:35.272 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-17T02:20:35.436 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-17T02:20:35.916 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-17T02:20:35.936 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-17T02:20:35.974 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-17T02:20:36.163 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-17T02:20:36.327 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package dump. 2024-08-17T02:20:36.345 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-17T02:20:36.385 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking dump (0.4b46-6) ... 2024-08-17T02:20:37.124 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package indent. 2024-08-17T02:20:37.141 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-17T02:20:37.173 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-17T02:20:37.478 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-17T02:20:37.494 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-17T02:20:37.559 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-17T02:20:37.887 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-17T02:20:37.905 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-17T02:20:37.938 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-17T02:20:37.943 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-17T02:20:37.946 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-17T02:20:38.057 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-17T02:20:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-17T02:20:38.241 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-17T02:20:38.315 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-17T02:20:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-17T02:20:38.652 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-17T02:20:38.669 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:38.680 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-17T02:20:38.710 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-17T02:20:38.979 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-17T02:20:39.007 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:39.007 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-17T02:20:39.054 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-17T02:20:39.382 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-17T02:20:39.399 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:39.440 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-17T02:20:39.466 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-17T02:20:39.470 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-17T02:20:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-17T02:20:39.793 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-17T02:20:39.812 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-17T02:20:39.851 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-17T02:20:40.086 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-17T02:20:40.171 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-17T02:20:40.191 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-17T02:20:40.226 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-17T02:20:40.229 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-17T02:20:40.626 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-17T02:20:40.639 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-17T02:20:40.642 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:40.684 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-17T02:20:40.771 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-17T02:20:41.012 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-17T02:20:41.030 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:41.070 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-17T02:20:41.292 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-17T02:20:41.310 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-17T02:20:41.322 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-17T02:20:41.341 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-17T02:20:41.381 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-17T02:20:41.392 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-17T02:20:41.703 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package dump. 2024-08-17T02:20:41.722 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-17T02:20:41.761 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking dump (0.4b46-6) ... 2024-08-17T02:20:41.860 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-17T02:20:41.877 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-17T02:20:41.909 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-17T02:20:42.564 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package modemmanager. 2024-08-17T02:20:42.584 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:42.593 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package indent. 2024-08-17T02:20:42.613 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-17T02:20:42.666 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-17T02:20:42.754 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-17T02:20:43.078 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-17T02:20:43.095 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-17T02:20:43.136 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-17T02:20:43.295 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package wpasupplicant. 2024-08-17T02:20:43.313 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-17T02:20:43.353 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-17T02:20:43.497 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-17T02:20:43.515 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-17T02:20:43.556 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-17T02:20:43.833 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-17T02:20:43.850 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-17T02:20:43.891 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-17T02:20:43.892 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package network-manager. 2024-08-17T02:20:43.910 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-17T02:20:43.977 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-17T02:20:44.270 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-17T02:20:44.287 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:44.328 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-17T02:20:44.622 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-17T02:20:44.639 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:44.680 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-17T02:20:44.717 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ppp. 2024-08-17T02:20:44.733 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-17T02:20:44.983 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-17T02:20:44.988 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-17T02:20:45.002 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:45.066 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-17T02:20:45.313 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package pptp-linux. 2024-08-17T02:20:45.332 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-17T02:20:45.371 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-17T02:20:45.428 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-17T02:20:45.444 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-17T02:20:45.511 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-17T02:20:45.725 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-17T02:20:45.744 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-17T02:20:45.792 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-17T02:20:45.847 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-17T02:20:45.866 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-17T02:20:45.930 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-17T02:20:46.011 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-17T02:20:46.031 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-17T02:20:46.069 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-17T02:20:46.300 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-17T02:20:46.320 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:46.373 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-17T02:20:46.472 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-17T02:20:46.489 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-17T02:20:46.541 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-17T02:20:46.694 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-17T02:20:46.713 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:46.752 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-17T02:20:46.830 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-17T02:20:47.029 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-17T02:20:47.046 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-17T02:20:47.088 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-17T02:20:47.145 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-17T02:20:47.271 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-17T02:20:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-17T02:20:47.608 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-17T02:20:47.624 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-17T02:20:47.666 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-17T02:20:47.789 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-17T02:20:47.789 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-17T02:20:48.235 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-17T02:20:48.361 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-17T02:20:48.487 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-17T02:20:48.538 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package modemmanager. 2024-08-17T02:20:48.555 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-17T02:20:48.607 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-17T02:20:48.646 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-17T02:20:48.772 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-17T02:20:48.898 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-17T02:20:49.254 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package wpasupplicant. 2024-08-17T02:20:49.271 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-17T02:20:49.312 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-17T02:20:49.835 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-17T02:20:49.876 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package network-manager. 2024-08-17T02:20:49.892 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-17T02:20:49.958 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-17T02:20:50.079 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up dump (0.4b46-6) ... 2024-08-17T02:20:50.174 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-17T02:20:50.239 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-17T02:20:50.365 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-17T02:20:50.491 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-17T02:20:50.617 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-17T02:20:50.708 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ppp. 2024-08-17T02:20:50.724 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-17T02:20:50.793 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-17T02:20:50.894 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-17T02:20:50.997 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-17T02:20:51.012 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-17T02:20:51.154 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-17T02:20:51.323 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-17T02:20:51.363 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package pptp-linux. 2024-08-17T02:20:51.379 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-17T02:20:51.421 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-17T02:20:51.448 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-17T02:20:51.818 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-17T02:20:51.835 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-17T02:20:51.892 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-17T02:20:52.152 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-17T02:20:52.168 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-17T02:20:52.183 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:20:52.183 INFO:teuthology.orchestra.run.smithi100.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-17T02:20:52.183 INFO:teuthology.orchestra.run.smithi100.stdout:which means they are currently configured by ifupdown: 2024-08-17T02:20:52.183 INFO:teuthology.orchestra.run.smithi100.stdout:- enp3s0f1 2024-08-17T02:20:52.183 INFO:teuthology.orchestra.run.smithi100.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-17T02:20:52.183 INFO:teuthology.orchestra.run.smithi100.stdout:remove their configuration from /etc/network/interfaces. 2024-08-17T02:20:52.183 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:20:52.210 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-17T02:20:52.349 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-17T02:20:52.622 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-17T02:20:52.639 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-17T02:20:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-17T02:20:52.691 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-17T02:20:53.034 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-17T02:20:53.212 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-17T02:20:53.344 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-17T02:20:53.662 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-17T02:20:53.782 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-17T02:20:53.837 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-17T02:20:53.979 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-17T02:20:54.105 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-17T02:20:54.183 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-17T02:20:54.231 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-17T02:20:54.341 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-17T02:20:54.584 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-17T02:20:54.584 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-17T02:20:54.650 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-17T02:20:54.650 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-17T02:20:55.042 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-17T02:20:55.161 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-17T02:20:55.193 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-17T02:20:55.295 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-17T02:20:55.328 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-17T02:20:55.420 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-17T02:20:55.454 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-17T02:20:55.572 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-17T02:20:55.613 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-17T02:20:55.739 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-17T02:20:55.857 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-17T02:20:56.700 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-17T02:20:56.970 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up dump (0.4b46-6) ... 2024-08-17T02:20:57.082 INFO:teuthology.orchestra.run.smithi123.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-17T02:20:57.147 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-17T02:20:57.272 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-17T02:20:57.423 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-17T02:20:57.524 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-17T02:20:57.667 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-17T02:20:57.785 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-17T02:20:57.953 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-17T02:20:58.079 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-17T02:20:58.247 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-17T02:20:58.398 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-17T02:20:59.071 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:20:59.071 INFO:teuthology.orchestra.run.smithi123.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-17T02:20:59.071 INFO:teuthology.orchestra.run.smithi123.stdout:which means they are currently configured by ifupdown: 2024-08-17T02:20:59.071 INFO:teuthology.orchestra.run.smithi123.stdout:- enp3s0f1 2024-08-17T02:20:59.071 INFO:teuthology.orchestra.run.smithi123.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-17T02:20:59.071 INFO:teuthology.orchestra.run.smithi123.stdout:remove their configuration from /etc/network/interfaces. 2024-08-17T02:20:59.071 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:20:59.224 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-17T02:20:59.552 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-17T02:20:59.934 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-17T02:21:00.035 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-17T02:21:00.089 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-17T02:21:00.601 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-17T02:21:00.752 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-17T02:21:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-17T02:21:00.893 DEBUG:teuthology.parallel:result is None 2024-08-17T02:21:00.895 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-17T02:21:01.046 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-17T02:21:01.172 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-17T02:21:01.274 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-17T02:21:01.520 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-17T02:21:01.521 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-17T02:21:02.043 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-17T02:21:02.152 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-17T02:21:02.278 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-17T02:21:02.454 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-17T02:21:02.747 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-17T02:21:06.683 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-17T02:21:06.775 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-17T02:21:07.597 INFO:teuthology.orchestra.run.smithi123.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-17T02:21:07.603 DEBUG:teuthology.parallel:result is None 2024-08-17T02:21:07.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:21:07.747 DEBUG:teuthology.orchestra.run.smithi100:> dpkg-query -W -f '${Version}' ceph 2024-08-17T02:21:07.771 INFO:teuthology.orchestra.run.smithi100.stdout:17.2.7-1375-g2577e813-1focal 2024-08-17T02:21:07.771 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-g2577e813-1focal 2024-08-17T02:21:07.771 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-g2577e813-1focal is installed. 2024-08-17T02:21:07.773 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:21:07.916 DEBUG:teuthology.orchestra.run.smithi123:> dpkg-query -W -f '${Version}' ceph 2024-08-17T02:21:07.940 INFO:teuthology.orchestra.run.smithi123.stdout:17.2.7-1375-g2577e813-1focal 2024-08-17T02:21:07.940 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-g2577e813-1focal 2024-08-17T02:21:07.940 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-g2577e813-1focal is installed. 2024-08-17T02:21:07.942 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-17T02:21:07.942 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:07.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-17T02:21:07.959 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:07.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-17T02:21:07.976 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-17T02:21:07.977 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:07.977 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-08-17T02:21:07.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-17T02:21:08.005 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:08.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-08-17T02:21:08.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-17T02:21:08.049 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-17T02:21:08.049 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:08.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-17T02:21:08.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-17T02:21:08.126 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:08.127 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-17T02:21:08.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-17T02:21:08.152 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-17T02:21:08.152 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:08.152 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-08-17T02:21:08.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-17T02:21:08.182 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:08.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-08-17T02:21:08.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-17T02:21:08.253 INFO:teuthology.run_tasks:Running task ceph... 2024-08-17T02:21:08.335 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-17T02:21:08.335 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-08-17T02:21:08.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-08-17T02:21:08.353 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-17T02:21:08.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-17T02:21:08.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-17T02:21:08.365 INFO:tasks.ceph:Creating extra log directories... 2024-08-17T02:21:08.365 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-17T02:21:08.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-17T02:21:08.430 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-08-17T02:21:08.431 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-08-17T02:21:08.431 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-08-17T02:21:08.431 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:08.431 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-17T02:21:08.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-17T02:21:08.479 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:08.479 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-17T02:21:08.486 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-17T02:21:08.570 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-08-17T02:21:08.570 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-17T02:21:08.571 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246', 'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7858246', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5995, 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e', 'targets': {'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '67db276914d29505c9707d04f3a3b8fb9d8aee4b', 'timestamp': '2024-08-17_01:02:08', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-17T02:21:08.572 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-17T02:21:08.576 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-17T02:21:08.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-08-17T02:21:08.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-08-17T02:21:08.599 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:08.600 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-08-17T02:21:08.643 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-17T02:21:08.643 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-08-17T02:21:08.649 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-17T02:21:08.650 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:21:08.650 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-08-17T02:21:08.650 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:21:08.650 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-08-17 02:16:55.000686748 +0000 2024-08-17T02:21:08.650 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-08-17 02:16:54.792691533 +0000 2024-08-17T02:21:08.650 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-08-17 02:16:54.792691533 +0000 2024-08-17T02:21:08.650 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-08-17T02:21:08.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-17T02:21:08.665 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-08-17T02:21:08.666 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-08-17T02:21:08.666 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000394645 s, 1.3 MB/s 2024-08-17T02:21:08.667 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-17T02:21:08.675 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-08-17T02:21:08.724 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-17T02:21:08.724 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:21:08.724 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-08-17T02:21:08.724 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:21:08.724 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-08-17 02:16:55.408677361 +0000 2024-08-17T02:21:08.724 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-08-17 02:16:55.200682147 +0000 2024-08-17T02:21:08.725 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-08-17 02:16:55.200682147 +0000 2024-08-17T02:21:08.725 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-08-17T02:21:08.725 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-17T02:21:08.739 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-08-17T02:21:08.739 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-08-17T02:21:08.739 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000417921 s, 1.2 MB/s 2024-08-17T02:21:08.739 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-17T02:21:08.747 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-08-17T02:21:08.795 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-17T02:21:08.796 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:21:08.796 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-08-17T02:21:08.796 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:21:08.796 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-08-17 02:16:55.864666869 +0000 2024-08-17T02:21:08.796 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-08-17 02:16:55.632672207 +0000 2024-08-17T02:21:08.796 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-08-17 02:16:55.632672207 +0000 2024-08-17T02:21:08.796 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-08-17T02:21:08.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-17T02:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-08-17T02:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-08-17T02:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000414371 s, 1.2 MB/s 2024-08-17T02:21:08.808 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-17T02:21:08.816 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-08-17T02:21:08.863 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-17T02:21:08.864 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:21:08.864 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-08-17T02:21:08.864 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:21:08.864 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-08-17 02:16:56.328656194 +0000 2024-08-17T02:21:08.864 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-08-17 02:16:56.116661071 +0000 2024-08-17T02:21:08.864 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-08-17 02:16:56.116661071 +0000 2024-08-17T02:21:08.864 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-08-17T02:21:08.864 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-17T02:21:08.875 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-08-17T02:21:08.875 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-08-17T02:21:08.875 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000379265 s, 1.3 MB/s 2024-08-17T02:21:08.877 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-17T02:21:08.884 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:08.884 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:08.885 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-08-17T02:21:08.892 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-17T02:21:08.892 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-08-17T02:21:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-17T02:21:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:21:08.899 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-08-17T02:21:08.899 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:21:08.900 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-08-17 02:16:57.705636702 +0000 2024-08-17T02:21:08.900 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-08-17 02:16:57.477641956 +0000 2024-08-17T02:21:08.900 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-08-17 02:16:57.477641956 +0000 2024-08-17T02:21:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-08-17T02:21:08.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-17T02:21:08.912 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-08-17T02:21:08.912 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-08-17T02:21:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000424534 s, 1.2 MB/s 2024-08-17T02:21:08.914 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-17T02:21:08.922 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-08-17T02:21:08.973 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-17T02:21:08.973 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:21:08.973 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-08-17T02:21:08.973 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:21:08.973 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-08-17 02:16:58.133626836 +0000 2024-08-17T02:21:08.973 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-08-17 02:16:57.917631814 +0000 2024-08-17T02:21:08.974 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-08-17 02:16:57.917631814 +0000 2024-08-17T02:21:08.974 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-08-17T02:21:08.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-17T02:21:08.985 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-08-17T02:21:08.985 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-08-17T02:21:08.985 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000421033 s, 1.2 MB/s 2024-08-17T02:21:08.986 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-17T02:21:08.995 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-08-17T02:21:09.045 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-17T02:21:09.045 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:21:09.045 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 369 Links: 1 2024-08-17T02:21:09.045 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:21:09.046 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-08-17 02:16:58.549617247 +0000 2024-08-17T02:21:09.046 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-08-17 02:16:58.329622318 +0000 2024-08-17T02:21:09.046 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-08-17 02:16:58.329622318 +0000 2024-08-17T02:21:09.046 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-08-17T02:21:09.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-17T02:21:09.061 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-08-17T02:21:09.061 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-08-17T02:21:09.061 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000392751 s, 1.3 MB/s 2024-08-17T02:21:09.062 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-17T02:21:09.070 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-08-17T02:21:09.121 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-17T02:21:09.121 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:21:09.121 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 375 Links: 1 2024-08-17T02:21:09.121 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:21:09.122 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-08-17 02:16:58.973607474 +0000 2024-08-17T02:21:09.122 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-08-17 02:16:58.753612545 +0000 2024-08-17T02:21:09.122 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-08-17 02:16:58.753612545 +0000 2024-08-17T02:21:09.122 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-08-17T02:21:09.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-17T02:21:09.134 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-08-17T02:21:09.134 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-08-17T02:21:09.134 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000364689 s, 1.4 MB/s 2024-08-17T02:21:09.135 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-17T02:21:09.145 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:09.145 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-17T02:21:09.145 INFO:tasks.ceph:Generating config... 2024-08-17T02:21:09.147 INFO:tasks.ceph:[client] client mount timeout = 600 2024-08-17T02:21:09.147 INFO:tasks.ceph:[client] debug client = 20 2024-08-17T02:21:09.147 INFO:tasks.ceph:[client] debug ms = 1 2024-08-17T02:21:09.147 INFO:tasks.ceph:[client] pid file = 2024-08-17T02:21:09.148 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-08-17T02:21:09.148 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-08-17T02:21:09.148 INFO:tasks.ceph:[mds] debug mds = 20 2024-08-17T02:21:09.148 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-08-17T02:21:09.148 INFO:tasks.ceph:[mds] debug ms = 1 2024-08-17T02:21:09.148 INFO:tasks.ceph:[mds] mds debug frag = True 2024-08-17T02:21:09.148 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-08-17T02:21:09.148 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-08-17T02:21:09.148 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-08-17T02:21:09.149 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-08-17T02:21:09.149 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-08-17T02:21:09.149 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-08-17T02:21:09.149 INFO:tasks.ceph:[mgr] debug client = 20 2024-08-17T02:21:09.149 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-08-17T02:21:09.149 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-17T02:21:09.149 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-17T02:21:09.149 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-17T02:21:09.150 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-17T02:21:09.150 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-17T02:21:09.150 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] bdev async discard = True 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-17T02:21:09.150 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-17T02:21:09.151 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-08-17T02:21:09.151 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-17T02:21:09.151 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-17T02:21:09.151 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-17T02:21:09.151 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-17T02:21:09.151 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-17T02:21:09.151 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-08-17T02:21:09.151 INFO:tasks.ceph:Setting up mon.a... 2024-08-17T02:21:09.151 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-08-17T02:21:09.188 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.keyring 2024-08-17T02:21:09.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-08-17T02:21:09.226 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-17T02:21:09.236 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.100'), ('mon.b', '172.21.15.123'), ('mon.c', '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]')] 2024-08-17T02:21:09.237 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.100,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-08-17T02:21:09.237 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:09.238 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-17T02:21:09.245 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.100 --add b 172.21.15.123 --addv c '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:09.324 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: generated fsid 147b3f96-a106-4648-b442-f3eda3a290c0 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = octopus 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 0 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:fsid 147b3f96-a106-4648-b442-f3eda3a290c0 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:last_changed 2024-08-17T02:21:09.325179+0000 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-08-17T02:21:09.325179+0000 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:min_mon_release 15 (octopus) 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:election_strategy: 1 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-08-17T02:21:09.325 INFO:teuthology.orchestra.run.smithi100.stdout:2: [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] mon.c 2024-08-17T02:21:09.326 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-08-17T02:21:09.327 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-17T02:21:09.333 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 147b3f96-a106-4648-b442-f3eda3a290c0... 2024-08-17T02:21:09.334 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-17T02:21:09.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-17T02:21:09.369 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-08-17T02:21:09.369 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-08-17T02:21:09.369 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-17T02:21:09.369 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-08-17T02:21:09.369 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-08-17T02:21:09.371 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-08-17T02:21:09.372 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.100,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 147b3f96-a106-4648-b442-f3eda3a290c0 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-08-17T02:21:09.373 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-08-17T02:21:09.374 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-08-17T02:21:09.375 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.376 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug finisher = 20 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-08-17T02:21:09.377 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-08-17T02:21:09.378 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-08-17T02:21:09.379 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-08-17T02:21:09.380 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-17T02:21:09.380 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-17T02:21:09.380 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-08-17T02:21:09.380 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-08-17T02:21:09.381 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-08-17T02:21:09.382 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-08-17T02:21:09.384 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-08-17T02:21:09.384 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-08-17T02:21:09.384 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-08-17T02:21:09.385 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-08-17T02:21:09.386 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.387 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.100,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 147b3f96-a106-4648-b442-f3eda3a290c0 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-08-17T02:21:09.388 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.389 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: bdev async discard = True 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: bdev enable discard = True 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore allocator = bitmap 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2024-08-17T02:21:09.390 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 1/20 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 1/20 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 4/10 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2024-08-17T02:21:09.391 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: debug finisher = 20 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.392 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-17T02:21:09.393 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-17T02:21:09.394 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-08-17T02:21:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2024-08-17T02:21:09.396 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds balancer = 20 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-08-17T02:21:09.397 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-08-17T02:21:09.397 INFO:tasks.ceph:Creating admin key on mon.a... 2024-08-17T02:21:09.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-08-17T02:21:09.448 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-17T02:21:09.449 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:09.449 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-08-17T02:21:09.454 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:09.454 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-08-17T02:21:09.503 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:21:09.503 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:09.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-17T02:21:09.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-17T02:21:09.527 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:09.527 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:09.533 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:21:09.533 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:09.534 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-17T02:21:09.534 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-17T02:21:09.559 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:09.559 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:09.608 INFO:tasks.ceph:Setting up mon nodes... 2024-08-17T02:21:09.609 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-17T02:21:09.609 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-08-17T02:21:09.655 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-08-17T02:21:09.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-08-17T02:21:09.703 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-08-17T02:21:09.706 INFO:tasks.ceph:Setting up mds nodes... 2024-08-17T02:21:09.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-08-17T02:21:09.750 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-08-17T02:21:09.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-08-17T02:21:09.767 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-08-17T02:21:09.852 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-08-17T02:21:09.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-08-17T02:21:09.869 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-08-17T02:21:09.915 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-08-17T02:21:09.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-08-17T02:21:09.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-08-17T02:21:09.977 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-08-17T02:21:09.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-08-17T02:21:10.034 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-17T02:21:10.035 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-17T02:21:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-08-17T02:21:10.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-08-17T02:21:10.161 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-08-17T02:21:10.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-08-17T02:21:10.211 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-08-17T02:21:10.226 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-08-17T02:21:10.258 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-08-17T02:21:10.269 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-17T02:21:10.270 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-17T02:21:10.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-08-17T02:21:10.287 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:10.287 INFO:tasks.ceph:role: osd.0 2024-08-17T02:21:10.287 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:21:10.287 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-17T02:21:10.841 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:21:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:21:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:21:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 2024-08-17T02:21:10.842 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:21:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:21:10.842 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:21:10.842 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-17T02:21:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:21:10.842 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:21:10.844 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-08-17T02:21:10.844 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-08-17T02:21:10.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-08-17T02:21:10.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:21:10.933 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-08-17T02:21:10.933 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-08-17T02:21:10.945 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:10.945 INFO:tasks.ceph:role: osd.1 2024-08-17T02:21:10.946 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:21:10.946 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:21:11.519 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:21:11.521 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-08-17T02:21:11.521 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-08-17T02:21:11.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-08-17T02:21:11.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:21:11.575 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-08-17T02:21:11.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-08-17T02:21:11.632 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:11.632 INFO:tasks.ceph:role: osd.2 2024-08-17T02:21:11.632 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:21:11.632 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-17T02:21:12.096 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:21:12.097 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:21:12.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-08-17T02:21:12.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-08-17T02:21:12.156 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-08-17T02:21:12.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:21:12.169 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-08-17T02:21:12.169 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-08-17T02:21:12.227 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:12.227 INFO:tasks.ceph:role: osd.3 2024-08-17T02:21:12.227 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:21:12.228 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-17T02:21:12.748 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:21:12.749 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:21:12.750 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-08-17T02:21:12.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-08-17T02:21:12.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-08-17T02:21:12.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:21:12.826 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-08-17T02:21:12.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:12.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:12.906+0000 7f1e5b09e3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-08-17T02:21:12.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:12.906+0000 7f1e5b09e3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-08-17T02:21:12.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:12.906+0000 7f1e5b09e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-17T02:21:12.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:12.906+0000 7f1e5b09e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-17T02:21:12.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:12.906+0000 7f1e5b09e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-17T02:21:12.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:12.910+0000 7f1e5b09e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-08-17T02:21:13.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:13.478+0000 7f1e5b09e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:14.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:14.250+0000 7f1e5b09e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:15.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:15.030+0000 7f1e5b09e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:15.885 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-08-17T02:21:15.901 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:15.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:15.982+0000 7ff6a19183c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-08-17T02:21:15.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:15.982+0000 7ff6a19183c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-08-17T02:21:15.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:15.982+0000 7ff6a19183c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-17T02:21:15.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:15.982+0000 7ff6a19183c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-17T02:21:15.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:15.982+0000 7ff6a19183c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-17T02:21:15.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:15.982+0000 7ff6a19183c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-08-17T02:21:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:16.542+0000 7ff6a19183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:17.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:17.326+0000 7ff6a19183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:18.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:18.106+0000 7ff6a19183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:18.953 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-08-17T02:21:18.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:19.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:19.046+0000 7fbeee0d43c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-08-17T02:21:19.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:19.046+0000 7fbeee0d43c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-08-17T02:21:19.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:19.046+0000 7fbeee0d43c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-17T02:21:19.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:19.046+0000 7fbeee0d43c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-17T02:21:19.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:19.046+0000 7fbeee0d43c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-17T02:21:19.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:19.046+0000 7fbeee0d43c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-08-17T02:21:19.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:19.606+0000 7fbeee0d43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:20.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:20.390+0000 7fbeee0d43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:21.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:21.178+0000 7fbeee0d43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:22.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-08-17T02:21:22.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:22.086+0000 7ffbafcf53c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-08-17T02:21:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:22.086+0000 7ffbafcf53c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-08-17T02:21:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:22.086+0000 7ffbafcf53c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-17T02:21:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:22.086+0000 7ffbafcf53c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-17T02:21:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:22.086+0000 7ffbafcf53c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-17T02:21:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:22.086+0000 7ffbafcf53c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-08-17T02:21:22.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:22.638+0000 7ffbafcf53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:23.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:23.426+0000 7ffbafcf53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:24.209+0000 7ffbafcf53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:25.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-08-17T02:21:25.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-08-17T02:21:25.146 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:25.146 INFO:tasks.ceph:role: osd.4 2024-08-17T02:21:25.146 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:21:25.146 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-17T02:21:25.694 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:21:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:21:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:21:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 2024-08-17T02:21:25.695 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:21:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:21:25.695 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:21:25.695 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-17T02:21:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:21:25.695 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:21:25.696 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-08-17T02:21:25.696 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-08-17T02:21:25.772 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-08-17T02:21:25.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:21:25.784 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: /sbin/restorecon: command not found 2024-08-17T02:21:25.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-08-17T02:21:25.794 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:25.795 INFO:tasks.ceph:role: osd.5 2024-08-17T02:21:25.795 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:21:25.795 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-17T02:21:26.319 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:21:26.320 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:21:26.321 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-08-17T02:21:26.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-08-17T02:21:26.381 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-08-17T02:21:26.392 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: /sbin/restorecon: command not found 2024-08-17T02:21:26.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:21:26.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-08-17T02:21:26.404 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:26.404 INFO:tasks.ceph:role: osd.6 2024-08-17T02:21:26.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:21:26.405 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-17T02:21:26.887 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:21:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:21:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:21:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 2024-08-17T02:21:26.887 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:21:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:21:26.887 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:21:26.888 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-17T02:21:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:21:26.888 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:21:26.889 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-08-17T02:21:26.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-08-17T02:21:26.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-08-17T02:21:26.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:21:26.952 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: /sbin/restorecon: command not found 2024-08-17T02:21:26.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-08-17T02:21:26.963 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:21:26.963 INFO:tasks.ceph:role: osd.7 2024-08-17T02:21:26.963 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:21:26.963 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-17T02:21:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:21:27.490 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:21:27.490 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:21:27.491 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 2024-08-17T02:21:27.491 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:21:27.491 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:21:27.491 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:21:27.491 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-17T02:21:27.491 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:21:27.491 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:21:27.493 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-08-17T02:21:27.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-08-17T02:21:27.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-08-17T02:21:27.564 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: /sbin/restorecon: command not found 2024-08-17T02:21:27.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:21:27.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:27.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:27.679+0000 7f6eedf103c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-08-17T02:21:27.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:27.679+0000 7f6eedf103c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-08-17T02:21:27.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:27.679+0000 7f6eedf103c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-17T02:21:27.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:27.679+0000 7f6eedf103c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-17T02:21:27.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:27.679+0000 7f6eedf103c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-17T02:21:27.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:27.687+0000 7f6eedf103c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-08-17T02:21:28.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:28.251+0000 7f6eedf103c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:29.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:29.023+0000 7f6eedf103c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:29.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:29.807+0000 7f6eedf103c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:30.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-08-17T02:21:30.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:30.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:30.715+0000 7f18516fe3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-08-17T02:21:30.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:30.715+0000 7f18516fe3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-08-17T02:21:30.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:30.715+0000 7f18516fe3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-17T02:21:30.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:30.715+0000 7f18516fe3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-17T02:21:30.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:30.715+0000 7f18516fe3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-17T02:21:30.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:30.715+0000 7f18516fe3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-08-17T02:21:31.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:31.271+0000 7f18516fe3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:32.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:32.063+0000 7f18516fe3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:32.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:32.843+0000 7f18516fe3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:33.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-08-17T02:21:33.758 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:33.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:33.791+0000 7f73703d53c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-08-17T02:21:33.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:33.791+0000 7f73703d53c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-08-17T02:21:33.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:33.791+0000 7f73703d53c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-17T02:21:33.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:33.791+0000 7f73703d53c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-17T02:21:33.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:33.791+0000 7f73703d53c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-17T02:21:33.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:33.795+0000 7f73703d53c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-08-17T02:21:34.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:34.355+0000 7f73703d53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:35.139+0000 7f73703d53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:35.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:35.919+0000 7f73703d53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:36.770 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-08-17T02:21:36.785 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:36.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:36.867+0000 7f0f804553c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-08-17T02:21:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:36.867+0000 7f0f804553c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-08-17T02:21:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:36.867+0000 7f0f804553c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-17T02:21:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:36.867+0000 7f0f804553c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-17T02:21:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:36.867+0000 7f0f804553c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-17T02:21:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:36.867+0000 7f0f804553c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-08-17T02:21:37.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:37.431+0000 7f0f804553c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:38.211+0000 7f0f804553c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:38.574 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:21:38.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:21:38.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:38.991+0000 7f0f804553c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:21:39.842 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-08-17T02:21:39.859 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-17T02:21:39.859 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:39.859 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-08-17T02:21:39.875 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:39.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-08-17T02:21:39.930 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:39.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-08-17T02:21:39.944 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:39.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-08-17T02:21:39.957 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:39.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-08-17T02:21:40.016 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:40.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-08-17T02:21:40.027 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:40.027 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-08-17T02:21:40.039 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.040 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-08-17T02:21:40.056 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-08-17T02:21:40.113 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-08-17T02:21:40.123 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.123 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-08-17T02:21:40.136 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.137 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-08-17T02:21:40.149 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-08-17T02:21:40.160 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-08-17T02:21:40.173 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:40.173 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-08-17T02:21:40.180 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:40.180 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-08-17T02:21:40.227 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.227 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-08-17T02:21:40.234 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:40.234 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-08-17T02:21:40.242 INFO:tasks.ceph:Adding keys to all mons... 2024-08-17T02:21:40.242 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-17T02:21:40.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAVCcBm2PEUJxAAoMsk1QkbyySm/5QCLvxh9A== 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAVCcBmoa7GLBAAEa267zm/LtpX8rM4H7yg8w== 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAVCcBmF/fPMhAAfa+QY0oBhYx9nTuka/+TdA== 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAYCcBmgnZCNhAA1nISJnYxJYmnry99/RzJgw== 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-08-17T02:21:40.253 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAbCcBmvSDFOhAAK0vSqY2GBGTvVn4FKXaIfA== 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAfCcBmQ/wSAxAAvRzMbFLmZWCyN6aUkIF6NA== 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAiCcBmc5FsBRAAldQsOEVDCiN/naewTHmTZQ== 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAVCcBmxt72KRAAwupnsLYFrW5/0eK7BF9Ecg== 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAVCcBmlTGVNhAAQ0osukqK1NPnmS0+QVksnA== 2024-08-17T02:21:40.254 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-08-17T02:21:40.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAVCcBmxqYBOhAA0HeZWzfPBTtfFOpQaxW/Hw== 2024-08-17T02:21:40.255 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2024-08-17T02:21:40.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAnCcBmNiSCKBAAB8xY3SkplnjexHfk3MGyGw== 2024-08-17T02:21:40.255 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2024-08-17T02:21:40.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAqCcBmknnQKhAA+i/GGSIpkfT001Fnt6aSJg== 2024-08-17T02:21:40.255 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2024-08-17T02:21:40.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAtCcBmocZgLxAAejHqqtvypf2E+OBjcKNS6A== 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAwCcBmPHC+MxAAop04f8WqhE1Ovi2r7HhxZw== 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAWCcBmxiJDBBAAOJ7HO5oxzLmoqqDQJJIT1w== 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout:[client.2] 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAWCcBmZOmsCRAAflJ9DnDrX79J5lpCR+YRSA== 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAWCcBmBW+sDBAAdA1JhVXsr5HnAiu/iIxSRg== 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout:[client.3] 2024-08-17T02:21:40.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAWCcBmkOhsDxAAqAgtA04spf0ioafXteRTSg== 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAVCcBm2PEUJxAAoMsk1QkbyySm/5QCLvxh9A== 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAVCcBmoa7GLBAAEa267zm/LtpX8rM4H7yg8w== 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAVCcBmF/fPMhAAfa+QY0oBhYx9nTuka/+TdA== 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAYCcBmgnZCNhAA1nISJnYxJYmnry99/RzJgw== 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-08-17T02:21:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAbCcBmvSDFOhAAK0vSqY2GBGTvVn4FKXaIfA== 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAfCcBmQ/wSAxAAvRzMbFLmZWCyN6aUkIF6NA== 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAiCcBmc5FsBRAAldQsOEVDCiN/naewTHmTZQ== 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAVCcBmxt72KRAAwupnsLYFrW5/0eK7BF9Ecg== 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAVCcBmlTGVNhAAQ0osukqK1NPnmS0+QVksnA== 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAVCcBmxqYBOhAA0HeZWzfPBTtfFOpQaxW/Hw== 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAnCcBmNiSCKBAAB8xY3SkplnjexHfk3MGyGw== 2024-08-17T02:21:40.259 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAqCcBmknnQKhAA+i/GGSIpkfT001Fnt6aSJg== 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtCcBmocZgLxAAejHqqtvypf2E+OBjcKNS6A== 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAwCcBmPHC+MxAAop04f8WqhE1Ovi2r7HhxZw== 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAWCcBmxiJDBBAAOJ7HO5oxzLmoqqDQJJIT1w== 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAWCcBmZOmsCRAAflJ9DnDrX79J5lpCR+YRSA== 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAWCcBmBW+sDBAAdA1JhVXsr5HnAiu/iIxSRg== 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2024-08-17T02:21:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAWCcBmkOhsDxAAqAgtA04spf0ioafXteRTSg== 2024-08-17T02:21:40.261 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-17T02:21:40.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-17T02:21:40.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:21:40.349 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:21:40.394 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:21:40.396 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:21:40.484 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:40.486 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:40.531 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:40.534 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:40.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:40.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:40.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:40.670 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:40.714 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-17T02:21:40.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-17T02:21:40.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:21:40.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:21:40.932 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:21:40.935 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:21:41.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:41.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:41.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:41.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:41.143 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:41.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:41.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:41.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:21:41.237 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:21:41.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:21:41.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:21:41.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:21:41.378 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:21:41.381 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:21:41.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:21:41.429 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:21:41.524 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-17T02:21:41.524 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-08-17T02:21:41.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-17T02:21:41.639 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-08-17T02:21:41.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-08-17T02:21:41.670 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-17T02:21:41.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-08-17T02:21:41.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-08-17T02:21:41.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-17T02:21:41.806 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-08-17T02:21:41.819 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:41.821 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:21:41.826 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-08-17T02:21:41.827 INFO:tasks.ceph.mon.a:Restarting daemon 2024-08-17T02:21:41.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-08-17T02:21:41.829 INFO:tasks.ceph.mon.a:Started 2024-08-17T02:21:41.830 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-17T02:21:41.830 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-17T02:21:41.833 INFO:tasks.ceph.mon.b:Started 2024-08-17T02:21:41.833 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-17T02:21:41.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-17T02:21:41.835 INFO:tasks.ceph.mon.c:Started 2024-08-17T02:21:41.835 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-08-17T02:21:41.836 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-08-17T02:21:41.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-08-17T02:21:41.838 INFO:tasks.ceph.mgr.y:Started 2024-08-17T02:21:41.838 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-08-17T02:21:41.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-08-17T02:21:41.840 INFO:tasks.ceph.mgr.x:Started 2024-08-17T02:21:41.840 INFO:tasks.ceph:Setting crush tunables to default 2024-08-17T02:21:41.840 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd crush tunables default 2024-08-17T02:21:41.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7649355700 1 Processor -- start 2024-08-17T02:21:41.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7649355700 1 -- start start 2024-08-17T02:21:41.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7649355700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:41.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7649355700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76440cba90 con 0x7f76441596d0 2024-08-17T02:21:41.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7649355700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76440cb160 con 0x7f7644067510 2024-08-17T02:21:41.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7649355700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76440b2b60 con 0x7f7644159300 2024-08-17T02:21:41.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f76427fc700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f76441596d0 0x7f7644066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46350/0 (socket says 172.21.15.100:46350) 2024-08-17T02:21:41.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 learned_addr learned my addr 172.21.15.100:0/4059250017 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:21:41.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:41.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-17T02:21:41.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:41.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:41.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:41.941+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-17T02:21:42.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.145+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:42.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.145+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-17T02:21:42.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.145+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:42.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.145+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:42.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.145+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-17T02:21:42.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.549+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:42.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.549+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-17T02:21:42.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.549+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:42.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.549+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:42.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:42.549+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-17T02:21:43.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:43.353+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:43.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:43.353+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-17T02:21:43.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:43.353+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:43.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:43.353+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:43.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:43.353+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-17T02:21:44.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 msgr2=0x7f76440b2230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:21:44.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7640ff9700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7644159300 0x7f76440b2230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:44.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 >> v1:172.21.15.123:6789/0 conn(0x7f7644067510 legacy=0x7f76440cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:44.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 >> v1:172.21.15.100:6789/0 conn(0x7f76441596d0 legacy=0x7f7644066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7640ff9700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7640ff9700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76440b2b60 con 0x7f7630003760 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7630008d50 con 0x7f7630006200 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-17T02:21:44.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:21:44.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:44.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:44.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:44.941+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-17T02:21:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-17T02:21:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-17T02:21:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:21:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:45.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:45.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.145+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:45.549+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f7642ffd700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f7642ffd700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:21:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:21:46.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:46.353+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-17T02:21:47.198 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:47.197+0000 7f75f03c6000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-17T02:21:47.205 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:47.202+0000 7f9133ef5000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-17T02:21:47.915 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:47.913+0000 7f75f03c6000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-17T02:21:47.924 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:47.922+0000 7f9133ef5000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-17T02:21:47.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630008690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:21:47.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 msgr2=0x7f7630005bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:21:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f7630003760 0x7f7630005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f763000c830 0x7f763000ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f76437fe700 1 -- 172.21.15.100:0/4059250017 reap_dead start 2024-08-17T02:21:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630006680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76440cb440 con 0x7f76441596d0 2024-08-17T02:21:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76440b2b60 con 0x7f763000c830 2024-08-17T02:21:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f7640ff9700 1 -- 172.21.15.100:0/4059250017 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7630008d50 con 0x7f7630006200 2024-08-17T02:21:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.100:3300/0 conn(0x7f763000c830 0x7f763000ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.941+0000 7f76437fe700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630006680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7641ffb700 1 -- 172.21.15.100:0/4059250017 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603883786 0 0) 0x7f76440cb440 con 0x7f76441596d0 2024-08-17T02:21:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7641ffb700 1 -- 172.21.15.100:0/4059250017 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f762c003570 con 0x7f76441596d0 2024-08-17T02:21:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 msgr2=0x7f7630006680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630006680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 >> v1:172.21.15.100:6789/0 conn(0x7f76441596d0 legacy=0x7f763000b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f76427fc700 1 -- 172.21.15.100:0/4059250017 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7630005c60 con 0x7f763000c830 2024-08-17T02:21:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f76427fc700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763000c830 0x7f763000ec90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7630003bd0 tx=0x7f7638009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb2cd2540ab4478 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7641ffb700 1 -- 172.21.15.100:0/4059250017 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638017460 con 0x7f763000c830 2024-08-17T02:21:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7641ffb700 1 -- 172.21.15.100:0/4059250017 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76380175c0 con 0x7f763000c830 2024-08-17T02:21:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7641ffb700 1 -- 172.21.15.100:0/4059250017 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638017940 con 0x7f763000c830 2024-08-17T02:21:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7649355700 1 -- 172.21.15.100:0/4059250017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763000c830 msgr2=0x7f763000ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7649355700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763000c830 0x7f763000ec90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7630003bd0 tx=0x7f7638009040 comp rx=0 tx=0).stop 2024-08-17T02:21:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7649355700 1 -- 172.21.15.100:0/4059250017 shutdown_connections 2024-08-17T02:21:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7649355700 1 --2- 172.21.15.100:0/4059250017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763000c830 0x7f763000ec90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7649355700 1 --2- 172.21.15.100:0/4059250017 >> v2:172.21.15.123:3300/0 conn(0x7f7630006200 0x7f7630006680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7649355700 1 -- 172.21.15.100:0/4059250017 >> 172.21.15.100:0/4059250017 conn(0x7f76440682c0 msgr2=0x7f76440686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7649355700 1 -- 172.21.15.100:0/4059250017 shutdown_connections 2024-08-17T02:21:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.945+0000 7f7649355700 1 -- 172.21.15.100:0/4059250017 wait complete. 2024-08-17T02:21:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.949+0000 7f7649355700 1 Processor -- start 2024-08-17T02:21:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.949+0000 7f7649355700 1 -- start start 2024-08-17T02:21:47.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7649355700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7630006200 0x7f764416ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:47.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7649355700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763000c830 0x7f76441506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:47.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7649355700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76441596d0 0x7f76440c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:47.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7649355700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76440c7c30 con 0x7f7630006200 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7649355700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764416b2b0 con 0x7f76441596d0 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7649355700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764415c1f0 con 0x7f763000c830 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7642ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7630006200 0x7f764416ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7642ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7630006200 0x7f764416ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37380/0 (socket says 172.21.15.100:37380) 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7642ffd700 1 -- 172.21.15.100:0/2143435969 learned_addr learned my addr 172.21.15.100:0/2143435969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f76437fe700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76441596d0 0x7f76440c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f76427fc700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763000c830 0x7f76441506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f76437fe700 1 -- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763000c830 msgr2=0x7f76441506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f76437fe700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763000c830 0x7f76441506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f76437fe700 1 -- 172.21.15.100:0/2143435969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7630006200 msgr2=0x7f764416ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f76437fe700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7630006200 0x7f764416ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f76437fe700 1 -- 172.21.15.100:0/2143435969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763800f040 con 0x7f76441596d0 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7642ffd700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7630006200 0x7f764416ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f76437fe700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76441596d0 0x7f76440c9580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76440d0160 tx=0x7f763c0016f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f762affd700 1 -- 172.21.15.100:0/2143435969 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c006c50 con 0x7f76441596d0 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f762affd700 1 -- 172.21.15.100:0/2143435969 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c011590 con 0x7f76441596d0 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f762affd700 1 -- 172.21.15.100:0/2143435969 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c011950 con 0x7f76441596d0 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76440c7300 con 0x7f76441596d0 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76440c78c0 con 0x7f76441596d0 2024-08-17T02:21:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f762affd700 1 -- 172.21.15.100:0/2143435969 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f763c01d460 con 0x7f76441596d0 2024-08-17T02:21:47.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.953+0000 7f762affd700 1 -- 172.21.15.100:0/2143435969 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f763c013740 con 0x7f76441596d0 2024-08-17T02:21:47.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.957+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76440c7de0 con 0x7f76441596d0 2024-08-17T02:21:47.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:47.961+0000 7f762affd700 1 -- 172.21.15.100:0/2143435969 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f76440c7de0 con 0x7f76441596d0 2024-08-17T02:21:47.967 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:47.965+0000 7f75f03c6000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-17T02:21:47.982 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:47.978+0000 7f9133ef5000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-17T02:21:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.033+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f76440645c0 con 0x7f76441596d0 2024-08-17T02:21:48.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f762affd700 1 -- 172.21.15.100:0/2143435969 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f763c01d960 con 0x7f76441596d0 2024-08-17T02:21:48.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76441596d0 msgr2=0x7f76440c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76441596d0 0x7f76440c9580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76440d0160 tx=0x7f763c0016f0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 shutdown_connections 2024-08-17T02:21:48.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76441596d0 0x7f76440c9580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763000c830 0x7f76441506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 --2- 172.21.15.100:0/2143435969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7630006200 0x7f764416ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 >> 172.21.15.100:0/2143435969 conn(0x7f76440682c0 msgr2=0x7f76440ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:48.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 shutdown_connections 2024-08-17T02:21:48.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:48.037+0000 7f7649355700 1 -- 172.21.15.100:0/2143435969 wait complete. 2024-08-17T02:21:48.042 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-08-17T02:21:48.052 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-08-17T02:21:48.052 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:48.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-08-17T02:21:48.063 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:48.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-08-17T02:21:48.074 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:48.074 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-08-17T02:21:48.077 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:48.077+0000 7f75f03c6000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-17T02:21:48.083 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-08-17T02:21:48.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-08-17T02:21:48.097 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:48.094+0000 7f9133ef5000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-17T02:21:48.138 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:48.138 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-08-17T02:21:48.153 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:48.153 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-08-17T02:21:48.209 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:48.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-08-17T02:21:48.219 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-17T02:21:48.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-08-17T02:21:48.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 1efac916-de27-46ba-9482-a634bf91666d 0 2024-08-17T02:21:48.294 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:48.293+0000 7f75f03c6000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-17T02:21:48.322 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:48.318+0000 7f9133ef5000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-17T02:21:48.343 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:48.341+0000 7f75f03c6000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-17T02:21:48.371 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:48.370+0000 7f9133ef5000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-17T02:21:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a63c2e700 1 Processor -- start 2024-08-17T02:21:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a63c2e700 1 -- start start 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a63c2e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c159300 0x7f5a5c0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a63c2e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5a5c1596d0 0x7f5a5c066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a63c2e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a5c15dbc0 0x7f5a5c067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a63c2e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5c0b2b60 con 0x7f5a5c15dbc0 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a63c2e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cba90 con 0x7f5a5c1596d0 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a63c2e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cb160 con 0x7f5a5c159300 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a611c9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5a5c1596d0 0x7f5a5c066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.374+0000 7f5a619ca700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c159300 0x7f5a5c0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a611c9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5a5c1596d0 0x7f5a5c066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52002/0 (socket says 172.21.15.123:52002) 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a611c9700 1 -- 172.21.15.123:0/351721934 learned_addr learned my addr 172.21.15.123:0/351721934 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a621cb700 1 --2- 172.21.15.123:0/351721934 >> v2:172.21.15.100:3300/0 conn(0x7f5a5c15dbc0 0x7f5a5c067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a619ca700 1 -- 172.21.15.123:0/351721934 >> v2:172.21.15.123:3300/0 conn(0x7f5a5c1596d0 msgr2=0x7f5a5c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a619ca700 1 --2- 172.21.15.123:0/351721934 >> v2:172.21.15.123:3300/0 conn(0x7f5a5c1596d0 0x7f5a5c066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a619ca700 1 -- 172.21.15.123:0/351721934 >> v2:172.21.15.100:3300/0 conn(0x7f5a5c15dbc0 msgr2=0x7f5a5c067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a619ca700 1 --2- 172.21.15.123:0/351721934 >> v2:172.21.15.100:3300/0 conn(0x7f5a5c15dbc0 0x7f5a5c067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a619ca700 1 -- 172.21.15.123:0/351721934 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0cb440 con 0x7f5a5c159300 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a621cb700 1 --2- 172.21.15.123:0/351721934 >> v2:172.21.15.100:3300/0 conn(0x7f5a5c15dbc0 0x7f5a5c067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a611c9700 1 --2- 172.21.15.123:0/351721934 >> v2:172.21.15.123:3300/0 conn(0x7f5a5c1596d0 0x7f5a5c066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a619ca700 1 --2- 172.21.15.123:0/351721934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c159300 0x7f5a5c0b2230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c089030 tx=0x7f5a580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2dccb8cefff668a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a609c8700 1 -- 172.21.15.123:0/351721934 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58011630 con 0x7f5a5c159300 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a609c8700 1 -- 172.21.15.123:0/351721934 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58011790 con 0x7f5a5c159300 2024-08-17T02:21:48.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a609c8700 1 -- 172.21.15.123:0/351721934 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58011b30 con 0x7f5a5c159300 2024-08-17T02:21:48.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 -- 172.21.15.123:0/351721934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c159300 msgr2=0x7f5a5c0b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 --2- 172.21.15.123:0/351721934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c159300 0x7f5a5c0b2230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c089030 tx=0x7f5a580091a0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 -- 172.21.15.123:0/351721934 shutdown_connections 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 --2- 172.21.15.123:0/351721934 >> v2:172.21.15.100:3300/0 conn(0x7f5a5c15dbc0 0x7f5a5c067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 --2- 172.21.15.123:0/351721934 >> v2:172.21.15.123:3300/0 conn(0x7f5a5c1596d0 0x7f5a5c066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 --2- 172.21.15.123:0/351721934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c159300 0x7f5a5c0b2230 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 -- 172.21.15.123:0/351721934 >> 172.21.15.123:0/351721934 conn(0x7f5a5c0682c0 msgr2=0x7f5a5c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 -- 172.21.15.123:0/351721934 shutdown_connections 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 -- 172.21.15.123:0/351721934 wait complete. 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 Processor -- start 2024-08-17T02:21:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.378+0000 7f5a63c2e700 1 -- start start 2024-08-17T02:21:48.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a5c159300 0x7f5a5c1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c1596d0 0x7f5a5c0c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a5c15dbc0 0x7f5a5c0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c15ab50 con 0x7f5a5c15dbc0 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0c4570 con 0x7f5a5c159300 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c15adc0 con 0x7f5a5c1596d0 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c1596d0 0x7f5a5c0c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a619ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a5c159300 0x7f5a5c1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c1596d0 0x7f5a5c0c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:33804/0 (socket says 172.21.15.123:33804) 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 -- 172.21.15.123:0/1408113596 learned_addr learned my addr 172.21.15.123:0/1408113596 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a621cb700 1 --2- 172.21.15.123:0/1408113596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a5c15dbc0 0x7f5a5c0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 -- 172.21.15.123:0/1408113596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a5c159300 msgr2=0x7f5a5c1556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 --2- 172.21.15.123:0/1408113596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a5c159300 0x7f5a5c1556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 -- 172.21.15.123:0/1408113596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a5c15dbc0 msgr2=0x7f5a5c0c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 --2- 172.21.15.123:0/1408113596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a5c15dbc0 0x7f5a5c0c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 -- 172.21.15.123:0/1408113596 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c0094f0 con 0x7f5a5c1596d0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a611c9700 1 --2- 172.21.15.123:0/1408113596 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c1596d0 0x7f5a5c0c3c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00ec50 tx=0x7f5a4c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a51ffb700 1 -- 172.21.15.123:0/1408113596 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00c850 con 0x7f5a5c1596d0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a51ffb700 1 -- 172.21.15.123:0/1408113596 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c00c9b0 con 0x7f5a5c1596d0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a51ffb700 1 -- 172.21.15.123:0/1408113596 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00cd30 con 0x7f5a5c1596d0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a58009420 con 0x7f5a5c1596d0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c155be0 con 0x7f5a5c1596d0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a51ffb700 1 -- 172.21.15.123:0/1408113596 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5a4c005a00 con 0x7f5a5c1596d0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a51ffb700 1 -- 172.21.15.123:0/1408113596 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5a4c013cc0 con 0x7f5a5c1596d0 2024-08-17T02:21:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.382+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c15b240 con 0x7f5a5c1596d0 2024-08-17T02:21:48.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.386+0000 7f5a51ffb700 1 -- 172.21.15.123:0/1408113596 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5a4c021060 con 0x7f5a5c1596d0 2024-08-17T02:21:48.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.462+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1efac916-de27-46ba-9482-a634bf91666d", "id": 0} v 0) v1 -- 0x7f5a5c0645c0 con 0x7f5a5c1596d0 2024-08-17T02:21:48.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.466+0000 7f5a51ffb700 1 -- 172.21.15.123:0/1408113596 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1efac916-de27-46ba-9482-a634bf91666d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a4c017070 con 0x7f5a5c1596d0 2024-08-17T02:21:48.470 INFO:teuthology.orchestra.run.smithi123.stdout:0 2024-08-17T02:21:48.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c1596d0 msgr2=0x7f5a5c0c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 --2- 172.21.15.123:0/1408113596 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c1596d0 0x7f5a5c0c3c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00ec50 tx=0x7f5a4c009250 comp rx=0 tx=0).stop 2024-08-17T02:21:48.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 shutdown_connections 2024-08-17T02:21:48.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 --2- 172.21.15.123:0/1408113596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a5c15dbc0 0x7f5a5c0c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 --2- 172.21.15.123:0/1408113596 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5c1596d0 0x7f5a5c0c3c70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 --2- 172.21.15.123:0/1408113596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a5c159300 0x7f5a5c1556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 >> 172.21.15.123:0/1408113596 conn(0x7f5a5c0682c0 msgr2=0x7f5a5c0b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 shutdown_connections 2024-08-17T02:21:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.470+0000 7f5a63c2e700 1 -- 172.21.15.123:0/1408113596 wait complete. 2024-08-17T02:21:48.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new f8ea8c11-8a12-4112-9405-41fb3b5f66e2 1 2024-08-17T02:21:48.523 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:48.521+0000 7f75f03c6000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-17T02:21:48.549 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:48.546+0000 7f9133ef5000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-17T02:21:48.585 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:48.585+0000 7f75f03c6000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-17T02:21:48.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.590+0000 7ff43c414700 1 Processor -- start 2024-08-17T02:21:48.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.590+0000 7ff43c414700 1 -- start start 2024-08-17T02:21:48.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff434156240 0x7ff4340b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4340b1740 0x7ff4340ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4340b1ba0 con 0x7ff434155e70 2024-08-17T02:21:48.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4340c9b90 con 0x7ff434156240 2024-08-17T02:21:48.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340ae120 con 0x7ff4340b1740 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43a9b1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4340b1740 0x7ff4340ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43a9b1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4340b1740 0x7ff4340ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:33808/0 (socket says 172.21.15.123:33808) 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43a9b1700 1 -- 172.21.15.123:0/3251501371 learned_addr learned my addr 172.21.15.123:0/3251501371 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff4399af700 1 --2- 172.21.15.123:0/3251501371 >> v2:172.21.15.100:3300/0 conn(0x7ff434156240 0x7ff4340b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43a9b1700 1 -- 172.21.15.123:0/3251501371 >> v2:172.21.15.100:3300/0 conn(0x7ff434156240 msgr2=0x7ff4340b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43a9b1700 1 --2- 172.21.15.123:0/3251501371 >> v2:172.21.15.100:3300/0 conn(0x7ff434156240 0x7ff4340b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43a9b1700 1 -- 172.21.15.123:0/3251501371 >> v1:172.21.15.100:6789/0 conn(0x7ff434155e70 legacy=0x7ff4340ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43a9b1700 1 -- 172.21.15.123:0/3251501371 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340ae3d0 con 0x7ff4340b1740 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43a9b1700 1 --2- 172.21.15.123:0/3251501371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4340b1740 0x7ff4340ca600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009f20 tx=0x7ff42c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57c0e35f12484f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff4391ae700 1 -- 172.21.15.123:0/3251501371 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c00e070 con 0x7ff4340b1740 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff4391ae700 1 -- 172.21.15.123:0/3251501371 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c00ba80 con 0x7ff4340b1740 2024-08-17T02:21:48.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff4391ae700 1 -- 172.21.15.123:0/3251501371 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c0092e0 con 0x7ff4340b1740 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 -- 172.21.15.123:0/3251501371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4340b1740 msgr2=0x7ff4340ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 --2- 172.21.15.123:0/3251501371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4340b1740 0x7ff4340ca600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009f20 tx=0x7ff42c009840 comp rx=0 tx=0).stop 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 -- 172.21.15.123:0/3251501371 shutdown_connections 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 --2- 172.21.15.123:0/3251501371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4340b1740 0x7ff4340ca600 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 --2- 172.21.15.123:0/3251501371 >> v2:172.21.15.100:3300/0 conn(0x7ff434156240 0x7ff4340b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 -- 172.21.15.123:0/3251501371 >> 172.21.15.123:0/3251501371 conn(0x7ff4340687a0 msgr2=0x7ff43406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 -- 172.21.15.123:0/3251501371 shutdown_connections 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.594+0000 7ff43c414700 1 -- 172.21.15.123:0/3251501371 wait complete. 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 Processor -- start 2024-08-17T02:21:48.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 -- start start 2024-08-17T02:21:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4340b1740 0x7ff4340aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff434155e70 0x7ff4340678c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff434156240 0x7ff4340a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff434169620 con 0x7ff434156240 2024-08-17T02:21:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340ac3d0 con 0x7ff4340b1740 2024-08-17T02:21:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43416a5b0 con 0x7ff434155e70 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff4399af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff434155e70 0x7ff4340678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4340b1740 0x7ff4340aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4340b1740 0x7ff4340aba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52014/0 (socket says 172.21.15.123:52014) 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 -- 172.21.15.123:0/2211104959 learned_addr learned my addr 172.21.15.123:0/2211104959 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a9b1700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff434156240 0x7ff4340a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 -- 172.21.15.123:0/2211104959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff434155e70 msgr2=0x7ff4340678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff434155e70 0x7ff4340678c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 -- 172.21.15.123:0/2211104959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff434156240 msgr2=0x7ff4340a9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff434156240 0x7ff4340a9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 -- 172.21.15.123:0/2211104959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c009ae0 con 0x7ff4340b1740 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff4399af700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff434155e70 0x7ff4340678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43a1b0700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4340b1740 0x7ff4340aba80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff43000c590 tx=0x7ff43000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff42a7fc700 1 -- 172.21.15.123:0/2211104959 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430011590 con 0x7ff4340b1740 2024-08-17T02:21:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff42a7fc700 1 -- 172.21.15.123:0/2211104959 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff430011710 con 0x7ff4340b1740 2024-08-17T02:21:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff434169830 con 0x7ff4340b1740 2024-08-17T02:21:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff42a7fc700 1 -- 172.21.15.123:0/2211104959 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430011ad0 con 0x7ff4340b1740 2024-08-17T02:21:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43416a090 con 0x7ff4340b1740 2024-08-17T02:21:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff42a7fc700 1 -- 172.21.15.123:0/2211104959 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff430012420 con 0x7ff4340b1740 2024-08-17T02:21:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff42a7fc700 1 -- 172.21.15.123:0/2211104959 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7ff430011e40 con 0x7ff4340b1740 2024-08-17T02:21:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.598+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43416a2b0 con 0x7ff4340b1740 2024-08-17T02:21:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.602+0000 7ff42a7fc700 1 -- 172.21.15.123:0/2211104959 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff430021060 con 0x7ff4340b1740 2024-08-17T02:21:48.607 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:48.606+0000 7f9133ef5000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-17T02:21:48.647 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:48.645+0000 7f75f03c6000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-17T02:21:48.667 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:48.666+0000 7f9133ef5000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-17T02:21:48.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.678+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f8ea8c11-8a12-4112-9405-41fb3b5f66e2", "id": 1} v 0) v1 -- 0x7ff4340645c0 con 0x7ff4340b1740 2024-08-17T02:21:48.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.686+0000 7ff42a7fc700 1 -- 172.21.15.123:0/2211104959 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8ea8c11-8a12-4112-9405-41fb3b5f66e2", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff430017070 con 0x7ff4340b1740 2024-08-17T02:21:48.690 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-08-17T02:21:48.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4340b1740 msgr2=0x7ff4340aba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4340b1740 0x7ff4340aba80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff43000c590 tx=0x7ff43000c8a0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 shutdown_connections 2024-08-17T02:21:48.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff434156240 0x7ff4340a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff434155e70 0x7ff4340678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 --2- 172.21.15.123:0/2211104959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4340b1740 0x7ff4340aba80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 >> 172.21.15.123:0/2211104959 conn(0x7ff4340687a0 msgr2=0x7ff43414ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:48.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 shutdown_connections 2024-08-17T02:21:48.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.690+0000 7ff43c414700 1 -- 172.21.15.123:0/2211104959 wait complete. 2024-08-17T02:21:48.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2 2 2024-08-17T02:21:48.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.802+0000 7fb5b18e1700 1 Processor -- start 2024-08-17T02:21:48.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.802+0000 7fb5b18e1700 1 -- start start 2024-08-17T02:21:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb5ac158ea0 0x7fb5ac159270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb5ac0a9860 0x7fb5ac0af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ac0aa900 con 0x7fb5ac0aea20 2024-08-17T02:21:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ac067490 con 0x7fb5ac158ea0 2024-08-17T02:21:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ac0a9cf0 con 0x7fb5ac0a9860 2024-08-17T02:21:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5abfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb5ac0a9860 0x7fb5ac0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b10e0700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb5ac0aea20 0x7fb5ac0686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47104/0 (socket says 172.21.15.123:47104) 2024-08-17T02:21:48.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b10e0700 1 -- 172.21.15.123:0/21028024 learned_addr learned my addr 172.21.15.123:0/21028024 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-08-17T02:21:48.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5abfff700 1 -- 172.21.15.123:0/21028024 >> v2:172.21.15.100:3300/0 conn(0x7fb5ac158ea0 msgr2=0x7fb5ac159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b08df700 1 --2- 172.21.15.123:0/21028024 >> v2:172.21.15.100:3300/0 conn(0x7fb5ac158ea0 0x7fb5ac159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5abfff700 1 --2- 172.21.15.123:0/21028024 >> v2:172.21.15.100:3300/0 conn(0x7fb5ac158ea0 0x7fb5ac159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5ab7fe700 1 -- 172.21.15.123:0/21028024 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674137384 0 0) 0x7fb5ac0aa900 con 0x7fb5ac0aea20 2024-08-17T02:21:48.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5abfff700 1 -- 172.21.15.123:0/21028024 >> v1:172.21.15.123:6789/0 conn(0x7fb5ac0aea20 legacy=0x7fb5ac0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5abfff700 1 -- 172.21.15.123:0/21028024 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac0a9fd0 con 0x7fb5ac0a9860 2024-08-17T02:21:48.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b08df700 1 --2- 172.21.15.123:0/21028024 >> v2:172.21.15.100:3300/0 conn(0x7fb5ac158ea0 0x7fb5ac159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:21:48.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5abfff700 1 --2- 172.21.15.123:0/21028024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0a9860 0x7fb5ac0af540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac06d2a0 tx=0x7fb5980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=820ba1938b0d8013 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:48.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5ab7fe700 1 -- 172.21.15.123:0/21028024 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59800b6b0 con 0x7fb5ac0a9860 2024-08-17T02:21:48.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5ab7fe700 1 -- 172.21.15.123:0/21028024 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59800b810 con 0x7fb5ac0a9860 2024-08-17T02:21:48.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5ab7fe700 1 -- 172.21.15.123:0/21028024 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59800bbb0 con 0x7fb5ac0a9860 2024-08-17T02:21:48.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 -- 172.21.15.123:0/21028024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0a9860 msgr2=0x7fb5ac0af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 --2- 172.21.15.123:0/21028024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0a9860 0x7fb5ac0af540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac06d2a0 tx=0x7fb5980091a0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 -- 172.21.15.123:0/21028024 shutdown_connections 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 --2- 172.21.15.123:0/21028024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0a9860 0x7fb5ac0af540 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 --2- 172.21.15.123:0/21028024 >> v2:172.21.15.100:3300/0 conn(0x7fb5ac158ea0 0x7fb5ac159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 -- 172.21.15.123:0/21028024 >> 172.21.15.123:0/21028024 conn(0x7fb5ac06e350 msgr2=0x7fb5ac06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 -- 172.21.15.123:0/21028024 shutdown_connections 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.806+0000 7fb5b18e1700 1 -- 172.21.15.123:0/21028024 wait complete. 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 Processor -- start 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 -- start start 2024-08-17T02:21:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5ac0a9860 0x7fb5ac0c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0aea20 0x7fb5ac0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ac158ea0 0x7fb5ac0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac0c54d0 con 0x7fb5ac0a9860 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac0ca800 con 0x7fb5ac0aea20 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac0c7b70 con 0x7fb5ac158ea0 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5abfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0aea20 0x7fb5ac0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b10e0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ac158ea0 0x7fb5ac0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5abfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0aea20 0x7fb5ac0c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52032/0 (socket says 172.21.15.123:52032) 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5abfff700 1 -- 172.21.15.123:0/2253133385 learned_addr learned my addr 172.21.15.123:0/2253133385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b08df700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5ac0a9860 0x7fb5ac0c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b10e0700 1 -- 172.21.15.123:0/2253133385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0aea20 msgr2=0x7fb5ac0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b10e0700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0aea20 0x7fb5ac0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b10e0700 1 -- 172.21.15.123:0/2253133385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5ac0a9860 msgr2=0x7fb5ac0c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b10e0700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5ac0a9860 0x7fb5ac0c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b10e0700 1 -- 172.21.15.123:0/2253133385 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb598009420 con 0x7fb5ac158ea0 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b08df700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5ac0a9860 0x7fb5ac0c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b10e0700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ac158ea0 0x7fb5ac0c6e00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400cbe0 tx=0x7fb5a40019a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5a8ff9700 1 -- 172.21.15.123:0/2253133385 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400a8b0 con 0x7fb5ac158ea0 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5a8ff9700 1 -- 172.21.15.123:0/2253133385 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a400aa30 con 0x7fb5ac158ea0 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac0c7e50 con 0x7fb5ac158ea0 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5a8ff9700 1 -- 172.21.15.123:0/2253133385 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400adf0 con 0x7fb5ac158ea0 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac0c8410 con 0x7fb5ac158ea0 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5a8ff9700 1 -- 172.21.15.123:0/2253133385 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb5a4007300 con 0x7fb5ac158ea0 2024-08-17T02:21:48.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5a8ff9700 1 -- 172.21.15.123:0/2253133385 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fb5a4012400 con 0x7fb5ac158ea0 2024-08-17T02:21:48.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.810+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ac0638f0 con 0x7fb5ac158ea0 2024-08-17T02:21:48.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.814+0000 7fb5a8ff9700 1 -- 172.21.15.123:0/2253133385 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fb5a40075b0 con 0x7fb5ac158ea0 2024-08-17T02:21:48.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.922+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2", "id": 2} v 0) v1 -- 0x7fb5ac14f460 con 0x7fb5ac158ea0 2024-08-17T02:21:48.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.930+0000 7fb5a8ff9700 1 -- 172.21.15.123:0/2253133385 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb5a40077f0 con 0x7fb5ac158ea0 2024-08-17T02:21:48.934 INFO:teuthology.orchestra.run.smithi123.stdout:2 2024-08-17T02:21:48.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ac158ea0 msgr2=0x7fb5ac0c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ac158ea0 0x7fb5ac0c6e00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400cbe0 tx=0x7fb5a40019a0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 shutdown_connections 2024-08-17T02:21:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ac158ea0 0x7fb5ac0c6e00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ac0aea20 0x7fb5ac0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 --2- 172.21.15.123:0/2253133385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5ac0a9860 0x7fb5ac0c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 >> 172.21.15.123:0/2253133385 conn(0x7fb5ac06e350 msgr2=0x7fb5ac0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 shutdown_connections 2024-08-17T02:21:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:48.934+0000 7fb5b18e1700 1 -- 172.21.15.123:0/2253133385 wait complete. 2024-08-17T02:21:48.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 820181f2-20e4-4352-86bf-b9c176577bda 3 2024-08-17T02:21:49.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25d37700 1 Processor -- start 2024-08-17T02:21:49.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25d37700 1 -- start start 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25d37700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fef20158ea0 0x7fef20159270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25d37700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef200a9860 0x7fef200af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25d37700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef200686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25d37700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef20067490 con 0x7fef20158ea0 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25d37700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200aa900 con 0x7fef200a9860 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25d37700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef200a9cf0 con 0x7fef200aea20 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef200a9860 0x7fef200af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef200a9860 0x7fef200af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52048/0 (socket says 172.21.15.123:52048) 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 -- 172.21.15.123:0/707917669 learned_addr learned my addr 172.21.15.123:0/707917669 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef24d35700 1 --2- 172.21.15.123:0/707917669 >> v2:172.21.15.100:3300/0 conn(0x7fef20158ea0 0x7fef20159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25536700 1 --2- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef200686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 -- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 msgr2=0x7fef200686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 --2- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef200686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 -- 172.21.15.123:0/707917669 >> v2:172.21.15.100:3300/0 conn(0x7fef20158ea0 msgr2=0x7fef20159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 --2- 172.21.15.123:0/707917669 >> v2:172.21.15.100:3300/0 conn(0x7fef20158ea0 0x7fef20159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 -- 172.21.15.123:0/707917669 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200a9fd0 con 0x7fef200a9860 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef25536700 1 --2- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef200686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.046+0000 7fef1ffff700 1 --2- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200af540 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef2006d2a0 tx=0x7fef08009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=718e19778a1d3fc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef1f7fe700 1 -- 172.21.15.123:0/707917669 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08009e70 con 0x7fef200a9860 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef1f7fe700 1 -- 172.21.15.123:0/707917669 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0800b860 con 0x7fef200a9860 2024-08-17T02:21:49.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef1f7fe700 1 -- 172.21.15.123:0/707917669 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0800bc00 con 0x7fef200a9860 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 -- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 msgr2=0x7fef200af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 --2- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200af540 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef2006d2a0 tx=0x7fef08009250 comp rx=0 tx=0).stop 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 -- 172.21.15.123:0/707917669 shutdown_connections 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 --2- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef200686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 --2- 172.21.15.123:0/707917669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200af540 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 --2- 172.21.15.123:0/707917669 >> v2:172.21.15.100:3300/0 conn(0x7fef20158ea0 0x7fef20159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 -- 172.21.15.123:0/707917669 >> 172.21.15.123:0/707917669 conn(0x7fef2006e350 msgr2=0x7fef2006b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 -- 172.21.15.123:0/707917669 shutdown_connections 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 -- 172.21.15.123:0/707917669 wait complete. 2024-08-17T02:21:49.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 Processor -- start 2024-08-17T02:21:49.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.050+0000 7fef25d37700 1 -- start start 2024-08-17T02:21:49.055 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:49.053+0000 7f75f03c6000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-17T02:21:49.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef20149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef20158ea0 0x7fef200c2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200c14c0 con 0x7fef20158ea0 2024-08-17T02:21:49.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200c16d0 con 0x7fef200a9860 2024-08-17T02:21:49.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef200c19b0 con 0x7fef200aea20 2024-08-17T02:21:49.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef20149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200c3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52058/0 (socket says 172.21.15.123:52058) 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 -- 172.21.15.123:0/3900364312 learned_addr learned my addr 172.21.15.123:0/3900364312 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25536700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef20158ea0 0x7fef200c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 -- 172.21.15.123:0/3900364312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 msgr2=0x7fef20149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef20149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 -- 172.21.15.123:0/3900364312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef20158ea0 msgr2=0x7fef200c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef20158ea0 0x7fef200c2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 -- 172.21.15.123:0/3900364312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080094f0 con 0x7fef200a9860 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25536700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef20158ea0 0x7fef200c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef24d35700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200c3690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef18009e50 tx=0x7fef1800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef1cff9700 1 -- 172.21.15.123:0/3900364312 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800cb50 con 0x7fef200a9860 2024-08-17T02:21:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef1cff9700 1 -- 172.21.15.123:0/3900364312 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1800f040 con 0x7fef200a9860 2024-08-17T02:21:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef200c1c90 con 0x7fef200a9860 2024-08-17T02:21:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef1cff9700 1 -- 172.21.15.123:0/3900364312 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18014480 con 0x7fef200a9860 2024-08-17T02:21:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef200c2250 con 0x7fef200a9860 2024-08-17T02:21:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef1cff9700 1 -- 172.21.15.123:0/3900364312 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fef18014a80 con 0x7fef200a9860 2024-08-17T02:21:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef1cff9700 1 -- 172.21.15.123:0/3900364312 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fef18014d30 con 0x7fef200a9860 2024-08-17T02:21:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.054+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef200638f0 con 0x7fef200a9860 2024-08-17T02:21:49.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.058+0000 7fef1cff9700 1 -- 172.21.15.123:0/3900364312 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fef18018020 con 0x7fef200a9860 2024-08-17T02:21:49.068 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:49.066+0000 7f9133ef5000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-17T02:21:49.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.130+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "820181f2-20e4-4352-86bf-b9c176577bda", "id": 3} v 0) v1 -- 0x7fef20066f40 con 0x7fef200a9860 2024-08-17T02:21:49.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef1cff9700 1 -- 172.21.15.123:0/3900364312 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "820181f2-20e4-4352-86bf-b9c176577bda", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef1801d770 con 0x7fef200a9860 2024-08-17T02:21:49.140 INFO:teuthology.orchestra.run.smithi123.stdout:3 2024-08-17T02:21:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 msgr2=0x7fef200c3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200c3690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef18009e50 tx=0x7fef1800b040 comp rx=0 tx=0).stop 2024-08-17T02:21:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 shutdown_connections 2024-08-17T02:21:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef20158ea0 0x7fef200c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef200aea20 0x7fef20149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 --2- 172.21.15.123:0/3900364312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef200a9860 0x7fef200c3690 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 >> 172.21.15.123:0/3900364312 conn(0x7fef2006e350 msgr2=0x7fef200688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 shutdown_connections 2024-08-17T02:21:49.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.138+0000 7fef25d37700 1 -- 172.21.15.123:0/3900364312 wait complete. 2024-08-17T02:21:49.153 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9 4 2024-08-17T02:21:49.169 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:49.169+0000 7f75f03c6000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-17T02:21:49.183 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:49.182+0000 7f9133ef5000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-17T02:21:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1f59e700 1 Processor -- start 2024-08-17T02:21:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1f59e700 1 -- start start 2024-08-17T02:21:49.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f200b0e90 0x7f5f2006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1f59e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200ad2c0 con 0x7f5f2006ec40 2024-08-17T02:21:49.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200b12c0 con 0x7f5f2013f880 2024-08-17T02:21:49.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2006f0c0 con 0x7f5f200b0e90 2024-08-17T02:21:49.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1e59c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5f2013f880 0x7f5f200b1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47116/0 (socket says 172.21.15.123:47116) 2024-08-17T02:21:49.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1e59c700 1 -- 172.21.15.123:0/814129939 learned_addr learned my addr 172.21.15.123:0/814129939 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-08-17T02:21:49.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1dd9b700 1 --2- 172.21.15.123:0/814129939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f200b0e90 0x7f5f2006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1d59a700 1 -- 172.21.15.123:0/814129939 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895828031 0 0) 0x7f5f200b12c0 con 0x7f5f2013f880 2024-08-17T02:21:49.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1d59a700 1 -- 172.21.15.123:0/814129939 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f08003570 con 0x7f5f2013f880 2024-08-17T02:21:49.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1dd9b700 1 -- 172.21.15.123:0/814129939 >> v1:172.21.15.123:6789/0 conn(0x7f5f2013f880 legacy=0x7f5f200b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1dd9b700 1 -- 172.21.15.123:0/814129939 >> v1:172.21.15.100:6789/0 conn(0x7f5f2006ec40 legacy=0x7f5f200ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1dd9b700 1 -- 172.21.15.123:0/814129939 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f200ac6d0 con 0x7f5f200b0e90 2024-08-17T02:21:49.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1dd9b700 1 --2- 172.21.15.123:0/814129939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f200b0e90 0x7f5f2006e700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009910 tx=0x7f5f140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b295f458db64f11a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.258+0000 7f5f1d59a700 1 -- 172.21.15.123:0/814129939 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400f6b0 con 0x7f5f200b0e90 2024-08-17T02:21:49.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1d59a700 1 -- 172.21.15.123:0/814129939 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400f810 con 0x7f5f200b0e90 2024-08-17T02:21:49.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1d59a700 1 -- 172.21.15.123:0/814129939 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400fb90 con 0x7f5f200b0e90 2024-08-17T02:21:49.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- 172.21.15.123:0/814129939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f200b0e90 msgr2=0x7f5f2006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 --2- 172.21.15.123:0/814129939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f200b0e90 0x7f5f2006e700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009910 tx=0x7f5f140091a0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- 172.21.15.123:0/814129939 shutdown_connections 2024-08-17T02:21:49.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 --2- 172.21.15.123:0/814129939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f200b0e90 0x7f5f2006e700 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- 172.21.15.123:0/814129939 >> 172.21.15.123:0/814129939 conn(0x7f5f200ca2c0 msgr2=0x7f5f2006dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- 172.21.15.123:0/814129939 shutdown_connections 2024-08-17T02:21:49.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- 172.21.15.123:0/814129939 wait complete. 2024-08-17T02:21:49.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 Processor -- start 2024-08-17T02:21:49.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- start start 2024-08-17T02:21:49.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2006ec40 0x7f5f20159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f200b0e90 0x7f5f200c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f2013f880 0x7f5f200c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2016a750 con 0x7f5f200b0e90 2024-08-17T02:21:49.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2016aa30 con 0x7f5f2013f880 2024-08-17T02:21:49.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2016ad10 con 0x7f5f2006ec40 2024-08-17T02:21:49.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2006ec40 0x7f5f20159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f2013f880 0x7f5f200c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2006ec40 0x7f5f20159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:33854/0 (socket says 172.21.15.123:33854) 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1e59c700 1 -- 172.21.15.123:0/2289797076 learned_addr learned my addr 172.21.15.123:0/2289797076 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1dd9b700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f200b0e90 0x7f5f200c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1ed9d700 1 -- 172.21.15.123:0/2289797076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2006ec40 msgr2=0x7f5f20159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1ed9d700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2006ec40 0x7f5f20159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1ed9d700 1 -- 172.21.15.123:0/2289797076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f200b0e90 msgr2=0x7f5f200c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1ed9d700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f200b0e90 0x7f5f200c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1ed9d700 1 -- 172.21.15.123:0/2289797076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14009420 con 0x7f5f2013f880 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1dd9b700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f200b0e90 0x7f5f200c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1ed9d700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f2013f880 0x7f5f200c5ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f10001eb0 tx=0x7f5f10007e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f0e7fc700 1 -- 172.21.15.123:0/2289797076 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10011590 con 0x7f5f2013f880 2024-08-17T02:21:49.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f0e7fc700 1 -- 172.21.15.123:0/2289797076 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f100116f0 con 0x7f5f2013f880 2024-08-17T02:21:49.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2016aff0 con 0x7f5f2013f880 2024-08-17T02:21:49.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f0e7fc700 1 -- 172.21.15.123:0/2289797076 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10011ab0 con 0x7f5f2013f880 2024-08-17T02:21:49.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.262+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f20159e70 con 0x7f5f2013f880 2024-08-17T02:21:49.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.266+0000 7f5f0e7fc700 1 -- 172.21.15.123:0/2289797076 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5f1001c440 con 0x7f5f2013f880 2024-08-17T02:21:49.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.266+0000 7f5f0e7fc700 1 -- 172.21.15.123:0/2289797076 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f5f10012930 con 0x7f5f2013f880 2024-08-17T02:21:49.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.266+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f200638f0 con 0x7f5f2013f880 2024-08-17T02:21:49.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.270+0000 7f5f0e7fc700 1 -- 172.21.15.123:0/2289797076 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5f10017070 con 0x7f5f2013f880 2024-08-17T02:21:49.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.358+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9", "id": 4} v 0) v1 -- 0x7f5f200cabc0 con 0x7f5f2013f880 2024-08-17T02:21:49.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f0e7fc700 1 -- 172.21.15.123:0/2289797076 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f10011e60 con 0x7f5f2013f880 2024-08-17T02:21:49.372 INFO:teuthology.orchestra.run.smithi123.stdout:4 2024-08-17T02:21:49.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f2013f880 msgr2=0x7f5f200c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f2013f880 0x7f5f200c5ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f10001eb0 tx=0x7f5f10007e20 comp rx=0 tx=0).stop 2024-08-17T02:21:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 shutdown_connections 2024-08-17T02:21:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f2013f880 0x7f5f200c5ad0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f200b0e90 0x7f5f200c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 --2- 172.21.15.123:0/2289797076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2006ec40 0x7f5f20159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 >> 172.21.15.123:0/2289797076 conn(0x7f5f200ca2c0 msgr2=0x7f5f200cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 shutdown_connections 2024-08-17T02:21:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.370+0000 7f5f1f59e700 1 -- 172.21.15.123:0/2289797076 wait complete. 2024-08-17T02:21:49.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new a54b7558-76f4-4ff3-8022-a2dbf14c709d 5 2024-08-17T02:21:49.460 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:49.457+0000 7f75f03c6000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-17T02:21:49.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff157fff700 1 Processor -- start 2024-08-17T02:21:49.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff157fff700 1 -- start start 2024-08-17T02:21:49.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff157fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff15815a680 0x7ff15806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff157fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15815a1f0 con 0x7ff1581570b0 2024-08-17T02:21:49.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff157fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff158153e00 con 0x7ff158153a30 2024-08-17T02:21:49.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff157fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580af0c0 con 0x7ff15815a680 2024-08-17T02:21:49.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff14ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff15815a680 0x7ff15806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff14ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff15815a680 0x7ff15806c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:33862/0 (socket says 172.21.15.123:33862) 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.490+0000 7ff14ffff700 1 -- 172.21.15.123:0/3295707467 learned_addr learned my addr 172.21.15.123:0/3295707467 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff14ffff700 1 -- 172.21.15.123:0/3295707467 >> v1:172.21.15.123:6789/0 conn(0x7ff158153a30 legacy=0x7ff1580ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff14ffff700 1 -- 172.21.15.123:0/3295707467 >> v1:172.21.15.100:6789/0 conn(0x7ff1581570b0 legacy=0x7ff1581575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff14ffff700 1 -- 172.21.15.123:0/3295707467 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580aede0 con 0x7ff15815a680 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff14ffff700 1 --2- 172.21.15.123:0/3295707467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff15815a680 0x7ff15806c200 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1400099e0 tx=0x7ff140009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e1b42d9c71b6c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff1567fc700 1 -- 172.21.15.123:0/3295707467 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000b860 con 0x7ff15815a680 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff1567fc700 1 -- 172.21.15.123:0/3295707467 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14000b9c0 con 0x7ff15815a680 2024-08-17T02:21:49.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff1567fc700 1 -- 172.21.15.123:0/3295707467 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000bd60 con 0x7ff15815a680 2024-08-17T02:21:49.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 -- 172.21.15.123:0/3295707467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff15815a680 msgr2=0x7ff15806c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 --2- 172.21.15.123:0/3295707467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff15815a680 0x7ff15806c200 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1400099e0 tx=0x7ff140009250 comp rx=0 tx=0).stop 2024-08-17T02:21:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 -- 172.21.15.123:0/3295707467 shutdown_connections 2024-08-17T02:21:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 --2- 172.21.15.123:0/3295707467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff15815a680 0x7ff15806c200 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 -- 172.21.15.123:0/3295707467 >> 172.21.15.123:0/3295707467 conn(0x7ff1580ca2b0 msgr2=0x7ff15806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 -- 172.21.15.123:0/3295707467 shutdown_connections 2024-08-17T02:21:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 -- 172.21.15.123:0/3295707467 wait complete. 2024-08-17T02:21:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 Processor -- start 2024-08-17T02:21:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.494+0000 7ff157fff700 1 -- start start 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff158153a30 0x7ff15816ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1581570b0 0x7ff15816ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1580c60e0 0x7ff15816a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580c7eb0 con 0x7ff1580c60e0 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15816aa80 con 0x7ff158153a30 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580c6570 con 0x7ff1581570b0 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1581570b0 0x7ff15816ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff156ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff158153a30 0x7ff15816ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1581570b0 0x7ff15816ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:33874/0 (socket says 172.21.15.123:33874) 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14ffff700 1 -- 172.21.15.123:0/255638619 learned_addr learned my addr 172.21.15.123:0/255638619 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff1577fe700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1580c60e0 0x7ff15816a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff156ffd700 1 -- 172.21.15.123:0/255638619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1581570b0 msgr2=0x7ff15816ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff156ffd700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1581570b0 0x7ff15816ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff156ffd700 1 -- 172.21.15.123:0/255638619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1580c60e0 msgr2=0x7ff15816a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff156ffd700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1580c60e0 0x7ff15816a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff156ffd700 1 -- 172.21.15.123:0/255638619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1400094f0 con 0x7ff158153a30 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14ffff700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1581570b0 0x7ff15816ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff156ffd700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff158153a30 0x7ff15816ace0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff14800c590 tx=0x7ff148007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14effd700 1 -- 172.21.15.123:0/255638619 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148011590 con 0x7ff158153a30 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14effd700 1 -- 172.21.15.123:0/255638619 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff148011710 con 0x7ff158153a30 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1580c6850 con 0x7ff158153a30 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14effd700 1 -- 172.21.15.123:0/255638619 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148011ad0 con 0x7ff158153a30 2024-08-17T02:21:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1580c6e10 con 0x7ff158153a30 2024-08-17T02:21:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14effd700 1 -- 172.21.15.123:0/255638619 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff14801c440 con 0x7ff158153a30 2024-08-17T02:21:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff14effd700 1 -- 172.21.15.123:0/255638619 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7ff1480129a0 con 0x7ff158153a30 2024-08-17T02:21:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.498+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1580638f0 con 0x7ff158153a30 2024-08-17T02:21:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.502+0000 7ff14effd700 1 -- 172.21.15.123:0/255638619 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff148017070 con 0x7ff158153a30 2024-08-17T02:21:49.511 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:49.510+0000 7f9133ef5000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-17T02:21:49.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.578+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a54b7558-76f4-4ff3-8022-a2dbf14c709d", "id": 5} v 0) v1 -- 0x7ff1581511f0 con 0x7ff158153a30 2024-08-17T02:21:49.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff14effd700 1 -- 172.21.15.123:0/255638619 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a54b7558-76f4-4ff3-8022-a2dbf14c709d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff14801c920 con 0x7ff158153a30 2024-08-17T02:21:49.593 INFO:teuthology.orchestra.run.smithi123.stdout:5 2024-08-17T02:21:49.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff158153a30 msgr2=0x7ff15816ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff158153a30 0x7ff15816ace0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff14800c590 tx=0x7ff148007ec0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 shutdown_connections 2024-08-17T02:21:49.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1580c60e0 0x7ff15816a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1581570b0 0x7ff15816ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 --2- 172.21.15.123:0/255638619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff158153a30 0x7ff15816ace0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 >> 172.21.15.123:0/255638619 conn(0x7ff1580ca2b0 msgr2=0x7ff158154020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 shutdown_connections 2024-08-17T02:21:49.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.590+0000 7ff157fff700 1 -- 172.21.15.123:0/255638619 wait complete. 2024-08-17T02:21:49.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new ce6e56ba-598d-4fb4-8841-69ec22272908 6 2024-08-17T02:21:49.625 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:49.625+0000 7f75f03c6000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-17T02:21:49.673 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:49.670+0000 7f9133ef5000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-17T02:21:49.691 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:49.689+0000 7f75f03c6000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-17T02:21:49.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376f59e700 1 Processor -- start 2024-08-17T02:21:49.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376f59e700 1 -- start start 2024-08-17T02:21:49.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f3770067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376f59e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3770158cc0 0x7f37701590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376f59e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37700cf670 con 0x7f37700cfb00 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376f59e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37700cea60 con 0x7f3770158cc0 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37700ced40 con 0x7f3770067030 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f3770067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f3770067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:33882/0 (socket says 172.21.15.123:33882) 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.706+0000 7f376e59c700 1 -- 172.21.15.123:0/276585475 learned_addr learned my addr 172.21.15.123:0/276585475 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376ed9d700 1 --2- 172.21.15.123:0/276585475 >> v2:172.21.15.100:3300/0 conn(0x7f3770158cc0 0x7f37701590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376e59c700 1 -- 172.21.15.123:0/276585475 >> v2:172.21.15.100:3300/0 conn(0x7f3770158cc0 msgr2=0x7f37701590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376e59c700 1 --2- 172.21.15.123:0/276585475 >> v2:172.21.15.100:3300/0 conn(0x7f3770158cc0 0x7f37701590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376e59c700 1 -- 172.21.15.123:0/276585475 >> v1:172.21.15.100:6789/0 conn(0x7f37700cfb00 legacy=0x7f37700cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376e59c700 1 -- 172.21.15.123:0/276585475 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37700cf020 con 0x7f3770067030 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376e59c700 1 --2- 172.21.15.123:0/276585475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f3770067400 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f37600099b0 tx=0x7f3760009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b4b15f58ff1b995 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376d59a700 1 -- 172.21.15.123:0/276585475 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376000b860 con 0x7f3770067030 2024-08-17T02:21:49.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376d59a700 1 -- 172.21.15.123:0/276585475 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376000b9c0 con 0x7f3770067030 2024-08-17T02:21:49.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376d59a700 1 -- 172.21.15.123:0/276585475 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376000bd60 con 0x7f3770067030 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 -- 172.21.15.123:0/276585475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 msgr2=0x7f3770067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 --2- 172.21.15.123:0/276585475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f3770067400 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f37600099b0 tx=0x7f3760009250 comp rx=0 tx=0).stop 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 -- 172.21.15.123:0/276585475 shutdown_connections 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 --2- 172.21.15.123:0/276585475 >> v2:172.21.15.100:3300/0 conn(0x7f3770158cc0 0x7f37701590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 --2- 172.21.15.123:0/276585475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f3770067400 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 -- 172.21.15.123:0/276585475 >> 172.21.15.123:0/276585475 conn(0x7f37700d0b50 msgr2=0x7f37700ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 -- 172.21.15.123:0/276585475 shutdown_connections 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 -- 172.21.15.123:0/276585475 wait complete. 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 Processor -- start 2024-08-17T02:21:49.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.710+0000 7f376f59e700 1 -- start start 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f377015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37700cfb00 0x7f377015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770158cc0 0x7f37700b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770169b90 con 0x7f3770158cc0 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770155330 con 0x7f37700cfb00 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377015bfe0 con 0x7f3770067030 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f377015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37700cfb00 0x7f377015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f377015b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:33890/0 (socket says 172.21.15.123:33890) 2024-08-17T02:21:49.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 -- 172.21.15.123:0/2763066854 learned_addr learned my addr 172.21.15.123:0/2763066854 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376ed9d700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770158cc0 0x7f37700b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 -- 172.21.15.123:0/2763066854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37700cfb00 msgr2=0x7f377015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37700cfb00 0x7f377015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 -- 172.21.15.123:0/2763066854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770158cc0 msgr2=0x7f37700b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770158cc0 0x7f37700b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 -- 172.21.15.123:0/2763066854 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37600094f0 con 0x7f3770067030 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376ed9d700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770158cc0 0x7f37700b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376e59c700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f377015b0a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37700cee80 tx=0x7f3760017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f375e7fc700 1 -- 172.21.15.123:0/2763066854 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760024440 con 0x7f3770067030 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f375e7fc700 1 -- 172.21.15.123:0/2763066854 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37600245a0 con 0x7f3770067030 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377015b710 con 0x7f3770067030 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f375e7fc700 1 -- 172.21.15.123:0/2763066854 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760024960 con 0x7f3770067030 2024-08-17T02:21:49.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377016a050 con 0x7f3770067030 2024-08-17T02:21:49.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f375e7fc700 1 -- 172.21.15.123:0/2763066854 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f3760024d50 con 0x7f3770067030 2024-08-17T02:21:49.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f375e7fc700 1 -- 172.21.15.123:0/2763066854 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f376001ade0 con 0x7f3770067030 2024-08-17T02:21:49.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.714+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37700638f0 con 0x7f3770067030 2024-08-17T02:21:49.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.718+0000 7f375e7fc700 1 -- 172.21.15.123:0/2763066854 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f3760015020 con 0x7f3770067030 2024-08-17T02:21:49.736 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:49.734+0000 7f9133ef5000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-17T02:21:49.745 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:49.745+0000 7f75f03c6000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-17T02:21:49.787 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:49.786+0000 7f9133ef5000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-17T02:21:49.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.790+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ce6e56ba-598d-4fb4-8841-69ec22272908", "id": 6} v 0) v1 -- 0x7f377015cc80 con 0x7f3770067030 2024-08-17T02:21:49.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f375e7fc700 1 -- 172.21.15.123:0/2763066854 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce6e56ba-598d-4fb4-8841-69ec22272908", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f376001a7e0 con 0x7f3770067030 2024-08-17T02:21:49.800 INFO:teuthology.orchestra.run.smithi123.stdout:6 2024-08-17T02:21:49.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 msgr2=0x7f377015b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f377015b0a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37700cee80 tx=0x7f3760017000 comp rx=0 tx=0).stop 2024-08-17T02:21:49.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 shutdown_connections 2024-08-17T02:21:49.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770158cc0 0x7f37700b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37700cfb00 0x7f377015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 --2- 172.21.15.123:0/2763066854 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3770067030 0x7f377015b0a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 >> 172.21.15.123:0/2763066854 conn(0x7f37700d0b50 msgr2=0x7f37700b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 shutdown_connections 2024-08-17T02:21:49.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.798+0000 7f376f59e700 1 -- 172.21.15.123:0/2763066854 wait complete. 2024-08-17T02:21:49.803 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:49.801+0000 7f75f03c6000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-17T02:21:49.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 220ff9d3-4fba-41b9-8c18-68374b5d22aa 7 2024-08-17T02:21:49.845 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:49.842+0000 7f9133ef5000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-17T02:21:49.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16b59e700 1 Processor -- start 2024-08-17T02:21:49.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16b59e700 1 -- start start 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa16c156d80 0x7fa16c0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16b59e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa16c157150 0x7fa16c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16b59e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c0ca700 con 0x7fa16c0cbd60 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16b59e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c0b21c0 con 0x7fa16c157150 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c0b24a0 con 0x7fa16c156d80 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16a59c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa16c156d80 0x7fa16c0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16a59c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa16c156d80 0x7fa16c0b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52100/0 (socket says 172.21.15.123:52100) 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16a59c700 1 -- 172.21.15.123:0/3204322458 learned_addr learned my addr 172.21.15.123:0/3204322458 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16ad9d700 1 --2- 172.21.15.123:0/3204322458 >> v2:172.21.15.100:3300/0 conn(0x7fa16c157150 0x7fa16c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16a59c700 1 -- 172.21.15.123:0/3204322458 >> v2:172.21.15.100:3300/0 conn(0x7fa16c157150 msgr2=0x7fa16c0cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16a59c700 1 --2- 172.21.15.123:0/3204322458 >> v2:172.21.15.100:3300/0 conn(0x7fa16c157150 0x7fa16c0cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16a59c700 1 -- 172.21.15.123:0/3204322458 >> v1:172.21.15.100:6789/0 conn(0x7fa16c0cbd60 legacy=0x7fa16c0ac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16a59c700 1 -- 172.21.15.123:0/3204322458 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0ca390 con 0x7fa16c156d80 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16a59c700 1 --2- 172.21.15.123:0/3204322458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c156d80 0x7fa16c0b1850 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa16c06c720 tx=0x7fa15c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=976a5ba0b1a44666 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16959a700 1 -- 172.21.15.123:0/3204322458 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00b860 con 0x7fa16c156d80 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16959a700 1 -- 172.21.15.123:0/3204322458 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c00b9c0 con 0x7fa16c156d80 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.902+0000 7fa16959a700 1 -- 172.21.15.123:0/3204322458 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00bd80 con 0x7fa16c156d80 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- 172.21.15.123:0/3204322458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c156d80 msgr2=0x7fa16c0b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 --2- 172.21.15.123:0/3204322458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c156d80 0x7fa16c0b1850 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa16c06c720 tx=0x7fa15c009250 comp rx=0 tx=0).stop 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- 172.21.15.123:0/3204322458 shutdown_connections 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 --2- 172.21.15.123:0/3204322458 >> v2:172.21.15.100:3300/0 conn(0x7fa16c157150 0x7fa16c0cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 --2- 172.21.15.123:0/3204322458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c156d80 0x7fa16c0b1850 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- 172.21.15.123:0/3204322458 >> 172.21.15.123:0/3204322458 conn(0x7fa16c06e310 msgr2=0x7fa16c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:49.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- 172.21.15.123:0/3204322458 shutdown_connections 2024-08-17T02:21:49.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- 172.21.15.123:0/3204322458 wait complete. 2024-08-17T02:21:49.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 Processor -- start 2024-08-17T02:21:49.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- start start 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c0cbd60 0x7fa16c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c156d80 0x7fa16c0aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa16c157150 0x7fa16c0c2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0a7460 con 0x7fa16c156d80 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0a7740 con 0x7fa16c0cbd60 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0aaf30 con 0x7fa16c157150 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c0cbd60 0x7fa16c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c0cbd60 0x7fa16c148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52106/0 (socket says 172.21.15.123:52106) 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16ad9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa16c157150 0x7fa16c0c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa16a59c700 1 -- 172.21.15.123:0/2157434175 learned_addr learned my addr 172.21.15.123:0/2157434175 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-17T02:21:49.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.906+0000 7fa169d9b700 1 --2- 172.21.15.123:0/2157434175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c156d80 0x7fa16c0aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:49.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16a59c700 1 -- 172.21.15.123:0/2157434175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa16c157150 msgr2=0x7fa16c0c2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16a59c700 1 --2- 172.21.15.123:0/2157434175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa16c157150 0x7fa16c0c2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16a59c700 1 -- 172.21.15.123:0/2157434175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c156d80 msgr2=0x7fa16c0aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:49.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16a59c700 1 --2- 172.21.15.123:0/2157434175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c156d80 0x7fa16c0aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:49.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16a59c700 1 -- 172.21.15.123:0/2157434175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c0094f0 con 0x7fa16c0cbd60 2024-08-17T02:21:49.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16a59c700 1 --2- 172.21.15.123:0/2157434175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c0cbd60 0x7fa16c148ff0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0b25e0 tx=0x7fa15c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:49.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa1627fc700 1 -- 172.21.15.123:0/2157434175 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c01f070 con 0x7fa16c0cbd60 2024-08-17T02:21:49.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa1627fc700 1 -- 172.21.15.123:0/2157434175 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c002f30 con 0x7fa16c0cbd60 2024-08-17T02:21:49.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c0ab210 con 0x7fa16c0cbd60 2024-08-17T02:21:49.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa1627fc700 1 -- 172.21.15.123:0/2157434175 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c011bd0 con 0x7fa16c0cbd60 2024-08-17T02:21:49.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c149530 con 0x7fa16c0cbd60 2024-08-17T02:21:49.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa1627fc700 1 -- 172.21.15.123:0/2157434175 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa15c023070 con 0x7fa16c0cbd60 2024-08-17T02:21:49.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa1627fc700 1 -- 172.21.15.123:0/2157434175 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fa15c01aa90 con 0x7fa16c0cbd60 2024-08-17T02:21:49.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c0638f0 con 0x7fa16c0cbd60 2024-08-17T02:21:49.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.910+0000 7fa1627fc700 1 -- 172.21.15.123:0/2157434175 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fa15c02a070 con 0x7fa16c0cbd60 2024-08-17T02:21:49.972 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:49.969+0000 7f75f03c6000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-17T02:21:49.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:49.990+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "220ff9d3-4fba-41b9-8c18-68374b5d22aa", "id": 7} v 0) v1 -- 0x7fa16c071360 con 0x7fa16c0cbd60 2024-08-17T02:21:50.003 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:50.002+0000 7f9133ef5000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-17T02:21:50.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa1627fc700 1 -- 172.21.15.123:0/2157434175 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "220ff9d3-4fba-41b9-8c18-68374b5d22aa", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa15c015020 con 0x7fa16c0cbd60 2024-08-17T02:21:50.004 INFO:teuthology.orchestra.run.smithi123.stdout:7 2024-08-17T02:21:50.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c0cbd60 msgr2=0x7fa16c148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:50.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 --2- 172.21.15.123:0/2157434175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c0cbd60 0x7fa16c148ff0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0b25e0 tx=0x7fa15c017000 comp rx=0 tx=0).stop 2024-08-17T02:21:50.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 shutdown_connections 2024-08-17T02:21:50.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 --2- 172.21.15.123:0/2157434175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa16c157150 0x7fa16c0c2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 --2- 172.21.15.123:0/2157434175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c156d80 0x7fa16c0aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 --2- 172.21.15.123:0/2157434175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16c0cbd60 0x7fa16c148ff0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 >> 172.21.15.123:0/2157434175 conn(0x7fa16c06e310 msgr2=0x7fa16c06ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:50.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 shutdown_connections 2024-08-17T02:21:50.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-17T02:21:50.002+0000 7fa16b59e700 1 -- 172.21.15.123:0/2157434175 wait complete. 2024-08-17T02:21:50.020 INFO:tasks.ceph.osd.0:Restarting daemon 2024-08-17T02:21:50.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-08-17T02:21:50.022 INFO:tasks.ceph.osd.0:Started 2024-08-17T02:21:50.023 INFO:tasks.ceph.osd.1:Restarting daemon 2024-08-17T02:21:50.023 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-08-17T02:21:50.025 INFO:tasks.ceph.osd.1:Started 2024-08-17T02:21:50.025 INFO:tasks.ceph.osd.2:Restarting daemon 2024-08-17T02:21:50.026 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-08-17T02:21:50.028 INFO:tasks.ceph.osd.2:Started 2024-08-17T02:21:50.029 INFO:tasks.ceph.osd.3:Restarting daemon 2024-08-17T02:21:50.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-08-17T02:21:50.032 INFO:tasks.ceph.osd.3:Started 2024-08-17T02:21:50.032 INFO:tasks.ceph.osd.4:Restarting daemon 2024-08-17T02:21:50.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-08-17T02:21:50.036 INFO:tasks.ceph.osd.4:Started 2024-08-17T02:21:50.036 INFO:tasks.ceph.osd.5:Restarting daemon 2024-08-17T02:21:50.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-08-17T02:21:50.038 INFO:tasks.ceph.osd.5:Started 2024-08-17T02:21:50.039 INFO:tasks.ceph.osd.6:Restarting daemon 2024-08-17T02:21:50.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-08-17T02:21:50.041 INFO:tasks.ceph.osd.6:Started 2024-08-17T02:21:50.041 INFO:tasks.ceph.osd.7:Restarting daemon 2024-08-17T02:21:50.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-08-17T02:21:50.044 INFO:tasks.ceph.osd.7:Started 2024-08-17T02:21:50.044 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:21:50.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.145+0000 7f4d8e0a9700 1 Processor -- start 2024-08-17T02:21:50.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d8e0a9700 1 -- start start 2024-08-17T02:21:50.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d8e0a9700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4d8815b390 0x7f4d88158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:50.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d8e0a9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d880ad9a0 0x7f4d880ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:50.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d8e0a9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8806d240 con 0x7f4d880ab800 2024-08-17T02:21:50.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d8e0a9700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8806dae0 con 0x7f4d8815b390 2024-08-17T02:21:50.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d8e0a9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d88158550 con 0x7f4d880ad9a0 2024-08-17T02:21:50.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d86ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4d8815b390 0x7f4d88158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:50.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d86ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4d8815b390 0x7f4d88158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37416/0 (socket says 172.21.15.100:37416) 2024-08-17T02:21:50.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d86ffd700 1 -- 172.21.15.100:0/99829816 learned_addr learned my addr 172.21.15.100:0/99829816 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:21:50.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d87fff700 1 --2- 172.21.15.100:0/99829816 >> v2:172.21.15.123:3300/0 conn(0x7f4d880ad9a0 0x7f4d880ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:50.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d86ffd700 1 -- 172.21.15.100:0/99829816 >> v2:172.21.15.123:3300/0 conn(0x7f4d880ad9a0 msgr2=0x7f4d880ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:50.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d86ffd700 1 --2- 172.21.15.100:0/99829816 >> v2:172.21.15.123:3300/0 conn(0x7f4d880ad9a0 0x7f4d880ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d86ffd700 1 -- 172.21.15.100:0/99829816 >> v1:172.21.15.123:6789/0 conn(0x7f4d880ab800 legacy=0x7f4d8815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:50.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d86ffd700 1 -- 172.21.15.100:0/99829816 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d880ae740 con 0x7f4d8815b390 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d86ffd700 1 --2- 172.21.15.100:0/99829816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88158b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d880cbe50 tx=0x7f4d78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d1f959034d52ee server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d867fc700 1 -- 172.21.15.100:0/99829816 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78009e70 con 0x7f4d8815b390 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d867fc700 1 -- 172.21.15.100:0/99829816 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7800b860 con 0x7f4d8815b390 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d867fc700 1 -- 172.21.15.100:0/99829816 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800bc00 con 0x7f4d8815b390 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/99829816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 msgr2=0x7f4d88158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.149+0000 7f4d8e0a9700 1 --2- 172.21.15.100:0/99829816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88158b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d880cbe50 tx=0x7f4d78009250 comp rx=0 tx=0).stop 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/99829816 shutdown_connections 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 --2- 172.21.15.100:0/99829816 >> v2:172.21.15.123:3300/0 conn(0x7f4d880ad9a0 0x7f4d880ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 --2- 172.21.15.100:0/99829816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88158b80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/99829816 >> 172.21.15.100:0/99829816 conn(0x7f4d880cc950 msgr2=0x7f4d88066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:50.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/99829816 shutdown_connections 2024-08-17T02:21:50.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/99829816 wait complete. 2024-08-17T02:21:50.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 Processor -- start 2024-08-17T02:21:50.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- start start 2024-08-17T02:21:50.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d880ab800 0x7f4d880c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:50.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d880ad9a0 0x7f4d8816daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:50.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:50.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8816d610 con 0x7f4d8815b390 2024-08-17T02:21:50.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d880c7840 con 0x7f4d880ab800 2024-08-17T02:21:50.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d880c7b20 con 0x7f4d880ad9a0 2024-08-17T02:21:50.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:50.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37432/0 (socket says 172.21.15.100:37432) 2024-08-17T02:21:50.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d877fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d880ab800 0x7f4d880c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:50.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d86ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d880ad9a0 0x7f4d8816daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:50.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 -- 172.21.15.100:0/3661194133 learned_addr learned my addr 172.21.15.100:0/3661194133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:21:50.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 -- 172.21.15.100:0/3661194133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d880ad9a0 msgr2=0x7f4d8816daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 --2- 172.21.15.100:0/3661194133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d880ad9a0 0x7f4d8816daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 -- 172.21.15.100:0/3661194133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d880ab800 msgr2=0x7f4d880c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 --2- 172.21.15.100:0/3661194133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d880ab800 0x7f4d880c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 -- 172.21.15.100:0/3661194133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d780094f0 con 0x7f4d8815b390 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d87fff700 1 --2- 172.21.15.100:0/3661194133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88154ba0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00ea50 tx=0x7f4d7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d6f7fe700 1 -- 172.21.15.100:0/3661194133 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00ee90 con 0x7f4d8815b390 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d6f7fe700 1 -- 172.21.15.100:0/3661194133 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00c850 con 0x7f4d8815b390 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d6f7fe700 1 -- 172.21.15.100:0/3661194133 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00cc10 con 0x7f4d8815b390 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8816cfc0 con 0x7f4d8815b390 2024-08-17T02:21:50.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.153+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d881550e0 con 0x7f4d8815b390 2024-08-17T02:21:50.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.157+0000 7f4d6f7fe700 1 -- 172.21.15.100:0/3661194133 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f4d7c005a00 con 0x7f4d8815b390 2024-08-17T02:21:50.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.157+0000 7f4d6f7fe700 1 -- 172.21.15.100:0/3661194133 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f4d7c013b50 con 0x7f4d8815b390 2024-08-17T02:21:50.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.161+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d880645c0 con 0x7f4d8815b390 2024-08-17T02:21:50.166 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:50.165+0000 7f75f03c6000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-17T02:21:50.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.165+0000 7f4d6f7fe700 1 -- 172.21.15.100:0/3661194133 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f4d7c021060 con 0x7f4d8815b390 2024-08-17T02:21:50.192 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:50.190+0000 7f9133ef5000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-17T02:21:50.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.233+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d8806efb0 con 0x7f4d8815b390 2024-08-17T02:21:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.233+0000 7f4d6f7fe700 1 -- 172.21.15.100:0/3661194133 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f4d7c017070 con 0x7f4d8815b390 2024-08-17T02:21:50.238 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:50.238 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":10,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:21:49.996480+0000","last_up_change":"0.000000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.233+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 msgr2=0x7f4d88154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.233+0000 7f4d8e0a9700 1 --2- 172.21.15.100:0/3661194133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88154ba0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00ea50 tx=0x7f4d7c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.237+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 shutdown_connections 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.237+0000 7f4d8e0a9700 1 --2- 172.21.15.100:0/3661194133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8815b390 0x7f4d88154ba0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.237+0000 7f4d8e0a9700 1 --2- 172.21.15.100:0/3661194133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d880ad9a0 0x7f4d8816daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.237+0000 7f4d8e0a9700 1 --2- 172.21.15.100:0/3661194133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d880ab800 0x7f4d880c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.237+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 >> 172.21.15.100:0/3661194133 conn(0x7f4d880cc950 msgr2=0x7f4d88155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.237+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 shutdown_connections 2024-08-17T02:21:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:50.237+0000 7f4d8e0a9700 1 -- 172.21.15.100:0/3661194133 wait complete. 2024-08-17T02:21:50.241 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:50.237+0000 7f75f03c6000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-17T02:21:50.248 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-08-17T02:21:50.249 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-17T02:21:50.257 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:50.254+0000 7f9133ef5000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-17T02:21:50.398 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-08-17T02:21:50.397+0000 7f91053863c0 -1 Falling back to public interface 2024-08-17T02:21:50.399 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-08-17T02:21:50.397+0000 7fdd9f8ce3c0 -1 Falling back to public interface 2024-08-17T02:21:50.399 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-08-17T02:21:50.397+0000 7f5db96c83c0 -1 Falling back to public interface 2024-08-17T02:21:50.400 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-08-17T02:21:50.398+0000 7f831bb303c0 -1 Falling back to public interface 2024-08-17T02:21:50.401 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-08-17T02:21:50.398+0000 7f95ac5123c0 -1 Falling back to public interface 2024-08-17T02:21:50.401 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-08-17T02:21:50.398+0000 7f831aaf9700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6806/0: (98) Address already in use 2024-08-17T02:21:50.415 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-08-17T02:21:50.414+0000 7f02d3ce93c0 -1 Falling back to public interface 2024-08-17T02:21:50.417 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:50.414+0000 7f9133ef5000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-17T02:21:50.419 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:50.417+0000 7f75f03c6000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-17T02:21:50.424 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-08-17T02:21:50.422+0000 7fc6d96ba3c0 -1 Falling back to public interface 2024-08-17T02:21:50.479 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:50.478+0000 7f9133ef5000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-17T02:21:50.490 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:50.489+0000 7f75f03c6000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-17T02:21:50.662 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-08-17T02:21:50.661+0000 7fbf4e5c03c0 -1 Falling back to public interface 2024-08-17T02:21:51.055 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-17T02:21:51.143 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.141+0000 7fcc6ffff700 1 Processor -- start 2024-08-17T02:21:51.144 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.141+0000 7fcc6ffff700 1 -- start start 2024-08-17T02:21:51.145 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.141+0000 7fcc6ffff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcc7013c230 0x7fcc700cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:51.145 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.141+0000 7fcc6ffff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc700c48b0 con 0x7fcc700cbdf0 2024-08-17T02:21:51.145 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.141+0000 7fcc6ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7015a770 con 0x7fcc7013be60 2024-08-17T02:21:51.145 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.141+0000 7fcc6ffff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7015b0b0 con 0x7fcc7013c230 2024-08-17T02:21:51.145 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6e7fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcc7013c230 0x7fcc700cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:51.145 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6f7fe700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcc700cbdf0 0x7fcc7006cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46046/0 (socket says 172.21.15.100:46046) 2024-08-17T02:21:51.145 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6e7fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcc7013c230 0x7fcc700cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37436/0 (socket says 172.21.15.100:37436) 2024-08-17T02:21:51.145 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6f7fe700 1 -- 172.21.15.100:0/3681818445 learned_addr learned my addr 172.21.15.100:0/3681818445 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6e7fc700 1 -- 172.21.15.100:0/3681818445 >> v1:172.21.15.123:6789/0 conn(0x7fcc7013be60 legacy=0x7fcc7015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6e7fc700 1 -- 172.21.15.100:0/3681818445 >> v1:172.21.15.100:6789/0 conn(0x7fcc700cbdf0 legacy=0x7fcc7006cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6e7fc700 1 -- 172.21.15.100:0/3681818445 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7006c1c0 con 0x7fcc7013c230 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6e7fc700 1 --2- 172.21.15.100:0/3681818445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc7013c230 0x7fcc700cb8b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcc640098e0 tx=0x7fcc640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a803ddca8d1181 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6dffb700 1 -- 172.21.15.100:0/3681818445 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64009e80 con 0x7fcc7013c230 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6dffb700 1 -- 172.21.15.100:0/3681818445 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6400b860 con 0x7fcc7013c230 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6dffb700 1 -- 172.21.15.100:0/3681818445 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400bbe0 con 0x7fcc7013c230 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 -- 172.21.15.100:0/3681818445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc7013c230 msgr2=0x7fcc700cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 --2- 172.21.15.100:0/3681818445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc7013c230 0x7fcc700cb8b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcc640098e0 tx=0x7fcc640091a0 comp rx=0 tx=0).stop 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 -- 172.21.15.100:0/3681818445 shutdown_connections 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 --2- 172.21.15.100:0/3681818445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc7013c230 0x7fcc700cb8b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:51.147 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 -- 172.21.15.100:0/3681818445 >> 172.21.15.100:0/3681818445 conn(0x7fcc70068360 msgr2=0x7fcc700cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:51.148 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 -- 172.21.15.100:0/3681818445 shutdown_connections 2024-08-17T02:21:51.148 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 -- 172.21.15.100:0/3681818445 wait complete. 2024-08-17T02:21:51.148 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 Processor -- start 2024-08-17T02:21:51.148 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.145+0000 7fcc6ffff700 1 -- start start 2024-08-17T02:21:51.151 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc700cbdf0 0x7fcc7013a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:51.151 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7013be60 0x7fcc7013dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:51.151 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7013c230 0x7fcc7016cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:51.151 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7013a8d0 con 0x7fcc700cbdf0 2024-08-17T02:21:51.151 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7013d7e0 con 0x7fcc7013c230 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7013cb10 con 0x7fcc7013be60 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc700cbdf0 0x7fcc7013a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc700cbdf0 0x7fcc7013a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37444/0 (socket says 172.21.15.100:37444) 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 -- 172.21.15.100:0/153637225 learned_addr learned my addr 172.21.15.100:0/153637225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6f7fe700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7013c230 0x7fcc7016cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6e7fc700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7013be60 0x7fcc7013dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 -- 172.21.15.100:0/153637225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7013be60 msgr2=0x7fcc7013dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7013be60 0x7fcc7013dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 -- 172.21.15.100:0/153637225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7013c230 msgr2=0x7fcc7016cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7013c230 0x7fcc7016cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 -- 172.21.15.100:0/153637225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64009420 con 0x7fcc700cbdf0 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6effd700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc700cbdf0 0x7fcc7013a390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c590 tx=0x7fcc6000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:51.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc56ffd700 1 -- 172.21.15.100:0/153637225 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60011590 con 0x7fcc700cbdf0 2024-08-17T02:21:51.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc56ffd700 1 -- 172.21.15.100:0/153637225 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc600116f0 con 0x7fcc700cbdf0 2024-08-17T02:21:51.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7016d6a0 con 0x7fcc700cbdf0 2024-08-17T02:21:51.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc56ffd700 1 -- 172.21.15.100:0/153637225 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60011ab0 con 0x7fcc700cbdf0 2024-08-17T02:21:51.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7013d190 con 0x7fcc700cbdf0 2024-08-17T02:21:51.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc56ffd700 1 -- 172.21.15.100:0/153637225 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fcc6001c440 con 0x7fcc700cbdf0 2024-08-17T02:21:51.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc56ffd700 1 -- 172.21.15.100:0/153637225 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fcc60011e20 con 0x7fcc700cbdf0 2024-08-17T02:21:51.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.149+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc70064ba0 con 0x7fcc700cbdf0 2024-08-17T02:21:51.155 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.153+0000 7fcc56ffd700 1 -- 172.21.15.100:0/153637225 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fcc60017070 con 0x7fcc700cbdf0 2024-08-17T02:21:51.182 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:51.178+0000 7f9133ef5000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-17T02:21:51.209 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.205+0000 7fcc56ffd700 1 -- 172.21.15.100:0/153637225 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 2) v1 ==== 39986+0+0 (secure 0 0 0) 0x7fcc6001c8d0 con 0x7fcc700cbdf0 2024-08-17T02:21:51.220 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.217+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc700c37f0 con 0x7fcc700cbdf0 2024-08-17T02:21:51.222 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc56ffd700 1 -- 172.21.15.100:0/153637225 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fcc60019c90 con 0x7fcc700cbdf0 2024-08-17T02:21:51.223 INFO:teuthology.misc.health.smithi100.stdout: 2024-08-17T02:21:51.223 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":10,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:21:49.996480+0000","last_up_change":"0.000000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:21:51.224 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc700cbdf0 msgr2=0x7fcc7013a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:51.224 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc700cbdf0 0x7fcc7013a390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c590 tx=0x7fcc6000c950 comp rx=0 tx=0).stop 2024-08-17T02:21:51.224 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 shutdown_connections 2024-08-17T02:21:51.224 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7013c230 0x7fcc7016cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:51.224 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7013be60 0x7fcc7013dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:51.224 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 --2- 172.21.15.100:0/153637225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc700cbdf0 0x7fcc7013a390 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:51.225 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 >> 172.21.15.100:0/153637225 conn(0x7fcc70068360 msgr2=0x7fcc700cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:51.225 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 shutdown_connections 2024-08-17T02:21:51.225 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:51.221+0000 7fcc6ffff700 1 -- 172.21.15.100:0/153637225 wait complete. 2024-08-17T02:21:51.234 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-08-17T02:21:51.233+0000 7f75f03c6000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-17T02:21:51.236 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-08-17T02:21:52.226 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:52.222+0000 7f911cf66700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-17T02:21:52.227 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-08-17T02:21:52.226+0000 7f911cf66700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-17T02:21:53.959 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-08-17T02:21:53.958+0000 7f831bb303c0 -1 osd.5 0 log_to_monitors true 2024-08-17T02:21:53.987 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-08-17T02:21:53.985+0000 7fdd9f8ce3c0 -1 osd.2 0 log_to_monitors true 2024-08-17T02:21:54.174 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-08-17T02:21:54.170+0000 7fc6d96ba3c0 -1 osd.7 0 log_to_monitors true 2024-08-17T02:21:54.187 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-08-17T02:21:54.185+0000 7f91053863c0 -1 osd.0 0 log_to_monitors true 2024-08-17T02:21:54.213 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-08-17T02:21:54.213+0000 7f5db96c83c0 -1 osd.3 0 log_to_monitors true 2024-08-17T02:21:54.235 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-08-17T02:21:54.234+0000 7f02d3ce93c0 -1 osd.6 0 log_to_monitors true 2024-08-17T02:21:54.242 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-08-17T02:21:54.238+0000 7f95ac5123c0 -1 osd.4 0 log_to_monitors true 2024-08-17T02:21:54.403 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-08-17T02:21:54.401+0000 7fbf4e5c03c0 -1 osd.1 0 log_to_monitors true 2024-08-17T02:21:56.864 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-08-17T02:21:56.861+0000 7fdd9c893700 -1 osd.2 0 waiting for initial osdmap 2024-08-17T02:21:56.868 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-08-17T02:21:56.865+0000 7f9100b26700 -1 osd.0 0 waiting for initial osdmap 2024-08-17T02:21:56.881 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-08-17T02:21:56.878+0000 7fc6d4e5a700 -1 osd.7 0 waiting for initial osdmap 2024-08-17T02:21:56.898 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-08-17T02:21:56.897+0000 7fdd93e11700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:21:56.902 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-08-17T02:21:56.901+0000 7f90f98c9700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:21:56.913 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-08-17T02:21:56.910+0000 7fc6cdbfd700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:21:56.925 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-08-17T02:21:56.922+0000 7f83172d0700 -1 osd.5 0 waiting for initial osdmap 2024-08-17T02:21:56.941 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-08-17T02:21:56.938+0000 7f8310073700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:21:57.810 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-08-17T02:21:57.806+0000 7f02cf489700 -1 osd.6 0 waiting for initial osdmap 2024-08-17T02:21:57.813 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-08-17T02:21:57.810+0000 7f02c822c700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:21:57.814 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-08-17T02:21:57.810+0000 7f95a7cb2700 -1 osd.4 0 waiting for initial osdmap 2024-08-17T02:21:57.818 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-08-17T02:21:57.814+0000 7f95a0a55700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:21:57.824 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-08-17T02:21:57.825+0000 7f5db4e68700 -1 osd.3 0 waiting for initial osdmap 2024-08-17T02:21:57.831 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-08-17T02:21:57.829+0000 7f5dadc0b700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:21:57.864 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-08-17T02:21:57.861+0000 7fbf4b585700 -1 osd.1 0 waiting for initial osdmap 2024-08-17T02:21:57.870 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-08-17T02:21:57.869+0000 7fbf42b03700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:21:58.044 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-17T02:21:58.148 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.145+0000 7f89f619b700 1 Processor -- start 2024-08-17T02:21:58.148 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.145+0000 7f89f619b700 1 -- start start 2024-08-17T02:21:58.149 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f0159030 0x7f89f00ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.150 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89f00c6030 0x7f89f00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.150 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f00cba70 con 0x7f89f0159400 2024-08-17T02:21:58.150 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00cbce0 con 0x7f89f00c6030 2024-08-17T02:21:58.150 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00cbfc0 con 0x7f89f0159030 2024-08-17T02:21:58.150 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f4998700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f89f0159400 0x7f89f00ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57928/0 (socket says 172.21.15.100:57928) 2024-08-17T02:21:58.150 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f4998700 1 -- 172.21.15.100:0/973836954 learned_addr learned my addr 172.21.15.100:0/973836954 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:21:58.150 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f599a700 1 --2- 172.21.15.100:0/973836954 >> v2:172.21.15.123:3300/0 conn(0x7f89f00c6030 0x7f89f00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.150 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f5199700 1 --2- 172.21.15.100:0/973836954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f0159030 0x7f89f00ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.151 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89e3fff700 1 -- 172.21.15.100:0/973836954 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779941786 0 0) 0x7f89f00cba70 con 0x7f89f0159400 2024-08-17T02:21:58.151 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89e3fff700 1 -- 172.21.15.100:0/973836954 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89dc003570 con 0x7f89f0159400 2024-08-17T02:21:58.151 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f599a700 1 -- 172.21.15.100:0/973836954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f0159030 msgr2=0x7f89f00ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:21:58.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f599a700 1 --2- 172.21.15.100:0/973836954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f0159030 0x7f89f00ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f599a700 1 -- 172.21.15.100:0/973836954 >> v1:172.21.15.100:6789/0 conn(0x7f89f0159400 legacy=0x7f89f00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f599a700 1 -- 172.21.15.100:0/973836954 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00cc2a0 con 0x7f89f00c6030 2024-08-17T02:21:58.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f599a700 1 --2- 172.21.15.100:0/973836954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f00c6030 0x7f89f00cd0c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009e60 tx=0x7f89e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be559566d033b533 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89e3fff700 1 -- 172.21.15.100:0/973836954 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800e070 con 0x7f89f00c6030 2024-08-17T02:21:58.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89e3fff700 1 -- 172.21.15.100:0/973836954 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e800bab0 con 0x7f89f00c6030 2024-08-17T02:21:58.152 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89e3fff700 1 -- 172.21.15.100:0/973836954 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800be70 con 0x7f89f00c6030 2024-08-17T02:21:58.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 -- 172.21.15.100:0/973836954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f00c6030 msgr2=0x7f89f00cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 --2- 172.21.15.100:0/973836954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f00c6030 0x7f89f00cd0c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009e60 tx=0x7f89e8009780 comp rx=0 tx=0).stop 2024-08-17T02:21:58.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 -- 172.21.15.100:0/973836954 shutdown_connections 2024-08-17T02:21:58.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 --2- 172.21.15.100:0/973836954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f00c6030 0x7f89f00cd0c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.153 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 --2- 172.21.15.100:0/973836954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f0159030 0x7f89f00ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.154 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.149+0000 7f89f619b700 1 -- 172.21.15.100:0/973836954 >> 172.21.15.100:0/973836954 conn(0x7f89f0087ff0 msgr2=0x7f89f00994e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:58.154 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.153+0000 7f89f619b700 1 -- 172.21.15.100:0/973836954 shutdown_connections 2024-08-17T02:21:58.154 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.153+0000 7f89f619b700 1 -- 172.21.15.100:0/973836954 wait complete. 2024-08-17T02:21:58.156 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.153+0000 7f89f619b700 1 Processor -- start 2024-08-17T02:21:58.156 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.153+0000 7f89f619b700 1 -- start start 2024-08-17T02:21:58.157 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00c6030 0x7f89f014fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89f0159030 0x7f89f014be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f0159400 0x7f89f0151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f014c3a0 con 0x7f89f0159030 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0151040 con 0x7f89f0159400 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f014cf70 con 0x7f89f00c6030 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89f0159030 0x7f89f014be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89f0159030 0x7f89f014be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36854/0 (socket says 172.21.15.100:36854) 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 -- 172.21.15.100:0/1119249374 learned_addr learned my addr 172.21.15.100:0/1119249374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f5199700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00c6030 0x7f89f014fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.158 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f599a700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f0159400 0x7f89f0151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.159 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 -- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00c6030 msgr2=0x7f89f014fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00c6030 0x7f89f014fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 -- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f0159400 msgr2=0x7f89f0151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f0159400 0x7f89f0151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 -- 172.21.15.100:0/1119249374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e8009a20 con 0x7f89f0159030 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f4998700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89f0159030 0x7f89f014be60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f89e400c590 tx=0x7f89e4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89e17fa700 1 -- 172.21.15.100:0/1119249374 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4005b30 con 0x7f89f0159030 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89e17fa700 1 -- 172.21.15.100:0/1119249374 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e4005cb0 con 0x7f89f0159030 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f01501b0 con 0x7f89f0159030 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89e17fa700 1 -- 172.21.15.100:0/1119249374 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4007320 con 0x7f89f0159030 2024-08-17T02:21:58.160 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f0150770 con 0x7f89f0159030 2024-08-17T02:21:58.161 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.157+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f014d0e0 con 0x7f89f0159030 2024-08-17T02:21:58.161 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.161+0000 7f89e17fa700 1 -- 172.21.15.100:0/1119249374 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f89e401b3b0 con 0x7f89f0159030 2024-08-17T02:21:58.161 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.161+0000 7f89e17fa700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89d4060800 0x7f89d4062cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.161 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.161+0000 7f89e17fa700 1 -- 172.21.15.100:0/1119249374 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f89e407ba60 con 0x7f89f0159030 2024-08-17T02:21:58.162 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.161+0000 7f89f5199700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89d4060800 0x7f89d4062cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.163 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.161+0000 7f89f5199700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89d4060800 0x7f89d4062cb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009db0 tx=0x7f89ec0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.171 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.169+0000 7f89e17fa700 1 -- 172.21.15.100:0/1119249374 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f89e404c3c0 con 0x7f89f0159030 2024-08-17T02:21:58.333 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.333+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89f0064ba0 con 0x7f89f0159030 2024-08-17T02:21:58.335 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.333+0000 7f89e17fa700 1 -- 172.21.15.100:0/1119249374 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10444 (secure 0 0 0) 0x7f89e4017090 con 0x7f89f0159030 2024-08-17T02:21:58.335 INFO:teuthology.misc.health.smithi100.stdout: 2024-08-17T02:21:58.335 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":14,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:21:57.907276+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:21:58.338 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89d4060800 msgr2=0x7f89d4062cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.338 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89d4060800 0x7f89d4062cb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009db0 tx=0x7f89ec0091a0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.338 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89f0159030 msgr2=0x7f89f014be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.339 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89f0159030 0x7f89f014be60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f89e400c590 tx=0x7f89e4007e00 comp rx=0 tx=0).stop 2024-08-17T02:21:58.339 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 shutdown_connections 2024-08-17T02:21:58.339 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89d4060800 0x7f89d4062cb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.339 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f0159400 0x7f89f0151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.339 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89f0159030 0x7f89f014be60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.340 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 --2- 172.21.15.100:0/1119249374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00c6030 0x7f89f014fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.340 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 >> 172.21.15.100:0/1119249374 conn(0x7f89f0087ff0 msgr2=0x7f89f006d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:58.340 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 shutdown_connections 2024-08-17T02:21:58.340 INFO:teuthology.misc.health.smithi100.stderr:2024-08-17T02:21:58.337+0000 7f89f619b700 1 -- 172.21.15.100:0/1119249374 wait complete. 2024-08-17T02:21:58.348 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-08-17T02:21:58.348 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-08-17T02:21:58.348 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-17T02:21:58.349 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-17T02:21:58.352 INFO:tasks.ceph.mds.a:Started 2024-08-17T02:21:58.353 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-17T02:21:58.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-17T02:21:58.355 INFO:tasks.ceph.mds.c:Started 2024-08-17T02:21:58.355 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-17T02:21:58.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-17T02:21:58.358 INFO:tasks.ceph.mds.b:Started 2024-08-17T02:21:58.361 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-17T02:21:58.361 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-17T02:21:58.364 INFO:tasks.ceph.mds.d:Started 2024-08-17T02:21:58.365 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-08-17T02:21:58.365 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:21:58.415 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-08-17T02:21:58.435 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-08-17T02:21:58.436 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2024-08-17T02:21:58.443 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2024-08-17T02:21:58.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.465+0000 7f38d582b700 1 Processor -- start 2024-08-17T02:21:58.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.465+0000 7f38d582b700 1 -- start start 2024-08-17T02:21:58.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.465+0000 7f38d582b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38d0159040 0x7f38d00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.465+0000 7f38d582b700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00cf190 con 0x7f38d015d5b0 2024-08-17T02:21:58.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.465+0000 7f38d582b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00ce5e0 con 0x7f38d00d0010 2024-08-17T02:21:58.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.465+0000 7f38d582b700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d015d9f0 con 0x7f38d0159040 2024-08-17T02:21:58.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.465+0000 7f38d4829700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38d0159040 0x7f38d00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d4829700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38d0159040 0x7f38d00cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36892/0 (socket says 172.21.15.100:36892) 2024-08-17T02:21:58.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d4829700 1 -- 172.21.15.100:0/286318606 learned_addr learned my addr 172.21.15.100:0/286318606 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d4829700 1 -- 172.21.15.100:0/286318606 >> v1:172.21.15.123:6789/0 conn(0x7f38d00d0010 legacy=0x7f38d015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d4829700 1 -- 172.21.15.100:0/286318606 >> v1:172.21.15.100:6789/0 conn(0x7f38d015d5b0 legacy=0x7f38d015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38cf7fe700 1 -- 172.21.15.100:0/286318606 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958923386 0 0) 0x7f38d00cf190 con 0x7f38d015d5b0 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d4829700 1 -- 172.21.15.100:0/286318606 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00ce860 con 0x7f38d0159040 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d4829700 1 --2- 172.21.15.100:0/286318606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00cf620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38d015db30 tx=0x7f38c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0d7454886b9fd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38cf7fe700 1 -- 172.21.15.100:0/286318606 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800e070 con 0x7f38d0159040 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38cf7fe700 1 -- 172.21.15.100:0/286318606 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c800ba50 con 0x7f38d0159040 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38cf7fe700 1 -- 172.21.15.100:0/286318606 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800be10 con 0x7f38d0159040 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 -- 172.21.15.100:0/286318606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 msgr2=0x7f38d00cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 --2- 172.21.15.100:0/286318606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00cf620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38d015db30 tx=0x7f38c8009840 comp rx=0 tx=0).stop 2024-08-17T02:21:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 -- 172.21.15.100:0/286318606 shutdown_connections 2024-08-17T02:21:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 --2- 172.21.15.100:0/286318606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00cf620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 -- 172.21.15.100:0/286318606 >> 172.21.15.100:0/286318606 conn(0x7f38d0068250 msgr2=0x7f38d0068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 -- 172.21.15.100:0/286318606 shutdown_connections 2024-08-17T02:21:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 -- 172.21.15.100:0/286318606 wait complete. 2024-08-17T02:21:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 Processor -- start 2024-08-17T02:21:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.469+0000 7f38d582b700 1 -- start start 2024-08-17T02:21:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00d0010 0x7f38d0149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d015d5b0 0x7f38d0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b2870 con 0x7f38d0159040 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b1c60 con 0x7f38d00d0010 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b1f40 con 0x7f38d015d5b0 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36896/0 (socket says 172.21.15.100:36896) 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 -- 172.21.15.100:0/3777501405 learned_addr learned my addr 172.21.15.100:0/3777501405 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d4829700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00d0010 0x7f38d0149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d502a700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d015d5b0 0x7f38d0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 -- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d015d5b0 msgr2=0x7f38d0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d015d5b0 0x7f38d0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 -- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00d0010 msgr2=0x7f38d0149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00d0010 0x7f38d0149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 -- 172.21.15.100:0/3777501405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8009ae0 con 0x7f38d0159040 2024-08-17T02:21:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d502a700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d015d5b0 0x7f38d0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:21:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38cffff700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00b1210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c4e0 tx=0x7f38c400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38ccff9700 1 -- 172.21.15.100:0/3777501405 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4011590 con 0x7f38d0159040 2024-08-17T02:21:58.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38ccff9700 1 -- 172.21.15.100:0/3777501405 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c4011710 con 0x7f38d0159040 2024-08-17T02:21:58.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d00b2220 con 0x7f38d0159040 2024-08-17T02:21:58.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38ccff9700 1 -- 172.21.15.100:0/3777501405 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4011ad0 con 0x7f38d0159040 2024-08-17T02:21:58.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d00afc20 con 0x7f38d0159040 2024-08-17T02:21:58.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.473+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d00b26a0 con 0x7f38d0159040 2024-08-17T02:21:58.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.477+0000 7f38ccff9700 1 -- 172.21.15.100:0/3777501405 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f38c401b3b0 con 0x7f38d0159040 2024-08-17T02:21:58.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.477+0000 7f38ccff9700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f38bc0607d0 0x7f38bc062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.477+0000 7f38ccff9700 1 -- 172.21.15.100:0/3777501405 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f38c407b950 con 0x7f38d0159040 2024-08-17T02:21:58.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.477+0000 7f38d4829700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f38bc0607d0 0x7f38bc062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.477+0000 7f38d4829700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f38bc0607d0 0x7f38bc062c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009ab0 tx=0x7f38c8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.485+0000 7f38ccff9700 1 -- 172.21.15.100:0/3777501405 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f38c404c350 con 0x7f38d0159040 2024-08-17T02:21:58.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.645+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38d00643a0 con 0x7f38d0159040 2024-08-17T02:21:58.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.649+0000 7f38ccff9700 1 -- 172.21.15.100:0/3777501405 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10444 (secure 0 0 0) 0x7f38c4017090 con 0x7f38d0159040 2024-08-17T02:21:58.649 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:58.649 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":14,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:21:57.907276+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.649+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f38bc0607d0 msgr2=0x7f38bc062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f38bc0607d0 0x7f38bc062c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009ab0 tx=0x7f38c8017000 comp rx=0 tx=0).stop 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 msgr2=0x7f38d00b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00b1210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c4e0 tx=0x7f38c400c7f0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 shutdown_connections 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f38bc0607d0 0x7f38bc062c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d015d5b0 0x7f38d0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d0159040 0x7f38d00b1210 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 --2- 172.21.15.100:0/3777501405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00d0010 0x7f38d0149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 >> 172.21.15.100:0/3777501405 conn(0x7f38d0068250 msgr2=0x7f38d00b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:58.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 shutdown_connections 2024-08-17T02:21:58.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.653+0000 7f38d582b700 1 -- 172.21.15.100:0/3777501405 wait complete. 2024-08-17T02:21:58.668 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-08-17T02:21:58.669 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:21:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad94b23700 1 Processor -- start 2024-08-17T02:21:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad94b23700 1 -- start start 2024-08-17T02:21:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad94b23700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fad900af030 0x7fad900a9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad94b23700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad900b00d0 con 0x7fad900a8f50 2024-08-17T02:21:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad94b23700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad90159d40 con 0x7fad901598a0 2024-08-17T02:21:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad94b23700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad900af4c0 con 0x7fad900af030 2024-08-17T02:21:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8e59c700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fad900a8f50 0x7fad900b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57950/0 (socket says 172.21.15.100:57950) 2024-08-17T02:21:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8e59c700 1 -- 172.21.15.100:0/2713684731 learned_addr learned my addr 172.21.15.100:0/2713684731 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:21:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8dd9b700 1 --2- 172.21.15.100:0/2713684731 >> v2:172.21.15.100:3300/0 conn(0x7fad900af030 0x7fad900a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8d59a700 1 -- 172.21.15.100:0/2713684731 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70704503 0 0) 0x7fad900b00d0 con 0x7fad900a8f50 2024-08-17T02:21:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8d59a700 1 -- 172.21.15.100:0/2713684731 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad78003570 con 0x7fad900a8f50 2024-08-17T02:21:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8dd9b700 1 -- 172.21.15.100:0/2713684731 >> v1:172.21.15.123:6789/0 conn(0x7fad901598a0 legacy=0x7fad900b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8dd9b700 1 -- 172.21.15.100:0/2713684731 >> v1:172.21.15.100:6789/0 conn(0x7fad900a8f50 legacy=0x7fad900b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8dd9b700 1 -- 172.21.15.100:0/2713684731 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad900af7a0 con 0x7fad900af030 2024-08-17T02:21:58.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8dd9b700 1 --2- 172.21.15.100:0/2713684731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900a9320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad840098e0 tx=0x7fad840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb2e83bb69044b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8d59a700 1 -- 172.21.15.100:0/2713684731 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400b640 con 0x7fad900af030 2024-08-17T02:21:58.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8d59a700 1 -- 172.21.15.100:0/2713684731 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8400b7a0 con 0x7fad900af030 2024-08-17T02:21:58.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.773+0000 7fad8d59a700 1 -- 172.21.15.100:0/2713684731 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400bb40 con 0x7fad900af030 2024-08-17T02:21:58.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 -- 172.21.15.100:0/2713684731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 msgr2=0x7fad900a9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 --2- 172.21.15.100:0/2713684731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900a9320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad840098e0 tx=0x7fad840091a0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 -- 172.21.15.100:0/2713684731 shutdown_connections 2024-08-17T02:21:58.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 --2- 172.21.15.100:0/2713684731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900a9320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 -- 172.21.15.100:0/2713684731 >> 172.21.15.100:0/2713684731 conn(0x7fad90066e00 msgr2=0x7fad90067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 -- 172.21.15.100:0/2713684731 shutdown_connections 2024-08-17T02:21:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 -- 172.21.15.100:0/2713684731 wait complete. 2024-08-17T02:21:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 Processor -- start 2024-08-17T02:21:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.777+0000 7fad94b23700 1 -- start start 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad94b23700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad900a8f50 0x7fad90144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad94b23700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900671d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad94b23700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad901598a0 0x7fad90067710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad94b23700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad90097320 con 0x7fad900af030 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad94b23700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad90097bc0 con 0x7fad901598a0 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad94b23700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9006a070 con 0x7fad900a8f50 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900671d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36916/0 (socket says 172.21.15.100:36916) 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 -- 172.21.15.100:0/4132880738 learned_addr learned my addr 172.21.15.100:0/4132880738 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:21:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8ed9d700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad901598a0 0x7fad90067710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8e59c700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad900a8f50 0x7fad90144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 -- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad900a8f50 msgr2=0x7fad90144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad900a8f50 0x7fad90144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 -- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad901598a0 msgr2=0x7fad90067710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad901598a0 0x7fad90067710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 -- 172.21.15.100:0/4132880738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad84009420 con 0x7fad900af030 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8e59c700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad900a8f50 0x7fad90144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad8dd9b700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900671d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad84009f80 tx=0x7fad84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad7e7fc700 1 -- 172.21.15.100:0/4132880738 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400bb20 con 0x7fad900af030 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad7e7fc700 1 -- 172.21.15.100:0/4132880738 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad84011420 con 0x7fad900af030 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad90097570 con 0x7fad900af030 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad7e7fc700 1 -- 172.21.15.100:0/4132880738 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad840117a0 con 0x7fad900af030 2024-08-17T02:21:58.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.781+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad90144fe0 con 0x7fad900af030 2024-08-17T02:21:58.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.785+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad900979f0 con 0x7fad900af030 2024-08-17T02:21:58.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.785+0000 7fad7e7fc700 1 -- 172.21.15.100:0/4132880738 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7fad84011b10 con 0x7fad900af030 2024-08-17T02:21:58.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.785+0000 7fad7e7fc700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fad700607d0 0x7fad70062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:21:58.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.785+0000 7fad7e7fc700 1 -- 172.21.15.100:0/4132880738 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fad8407dd90 con 0x7fad900af030 2024-08-17T02:21:58.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.785+0000 7fad8e59c700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fad700607d0 0x7fad70062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:21:58.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.785+0000 7fad8e59c700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fad700607d0 0x7fad70062c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fad88005c50 tx=0x7fad88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:21:58.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.793+0000 7fad7e7fc700 1 -- 172.21.15.100:0/4132880738 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fad84081030 con 0x7fad900af030 2024-08-17T02:21:58.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.989+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad900645c0 con 0x7fad900af030 2024-08-17T02:21:58.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.989+0000 7fad7e7fc700 1 -- 172.21.15.100:0/4132880738 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7fad84015500 con 0x7fad900af030 2024-08-17T02:21:58.991 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:21:58.992 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-08-17T02:21:58.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fad700607d0 msgr2=0x7fad70062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fad700607d0 0x7fad70062c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fad88005c50 tx=0x7fad88009040 comp rx=0 tx=0).stop 2024-08-17T02:21:58.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 msgr2=0x7fad900671d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:21:58.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900671d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad84009f80 tx=0x7fad84017000 comp rx=0 tx=0).stop 2024-08-17T02:21:58.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 shutdown_connections 2024-08-17T02:21:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fad700607d0 0x7fad70062c80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad901598a0 0x7fad90067710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad900af030 0x7fad900671d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 --2- 172.21.15.100:0/4132880738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad900a8f50 0x7fad90144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:21:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 >> 172.21.15.100:0/4132880738 conn(0x7fad90066e00 msgr2=0x7fad9006abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:21:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 shutdown_connections 2024-08-17T02:21:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:21:58.993+0000 7fad94b23700 1 -- 172.21.15.100:0/4132880738 wait complete. 2024-08-17T02:21:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-08-17T02:22:01.008 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:01.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f917c2700 1 Processor -- start 2024-08-17T02:22:01.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f917c2700 1 -- start start 2024-08-17T02:22:01.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f917c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b14a0 0x7f1f8c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f917c2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1f8c0cb390 0x7f1f8c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f917c2700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c0cbaa0 con 0x7f1f8c154900 2024-08-17T02:22:01.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f917c2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c068790 con 0x7f1f8c0cb390 2024-08-17T02:22:01.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f917c2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c1583f0 con 0x7f1f8c0b14a0 2024-08-17T02:22:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8bfff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1f8c154900 0x7f1f8c0b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57964/0 (socket says 172.21.15.100:57964) 2024-08-17T02:22:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8bfff700 1 -- 172.21.15.100:0/759143601 learned_addr learned my addr 172.21.15.100:0/759143601 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f90fc1700 1 --2- 172.21.15.100:0/759143601 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c0cb390 0x7f1f8c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8b7fe700 1 --2- 172.21.15.100:0/759143601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b14a0 0x7f1f8c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8affd700 1 -- 172.21.15.100:0/759143601 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450419958 0 0) 0x7f1f8c0cbaa0 con 0x7f1f8c154900 2024-08-17T02:22:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8affd700 1 -- 172.21.15.100:0/759143601 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f78003570 con 0x7f1f8c154900 2024-08-17T02:22:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8b7fe700 1 -- 172.21.15.100:0/759143601 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c0cb390 msgr2=0x7f1f8c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8b7fe700 1 --2- 172.21.15.100:0/759143601 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c0cb390 0x7f1f8c160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8b7fe700 1 -- 172.21.15.100:0/759143601 >> v1:172.21.15.100:6789/0 conn(0x7f1f8c154900 legacy=0x7f1f8c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8b7fe700 1 -- 172.21.15.100:0/759143601 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c157630 con 0x7f1f8c0b14a0 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f90fc1700 1 --2- 172.21.15.100:0/759143601 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c0cb390 0x7f1f8c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8b7fe700 1 --2- 172.21.15.100:0/759143601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b14a0 0x7f1f8c0cae50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009e60 tx=0x7f1f7c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=872230dd8ee5ceaf server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8affd700 1 -- 172.21.15.100:0/759143601 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00e070 con 0x7f1f8c0b14a0 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.121+0000 7f1f8affd700 1 -- 172.21.15.100:0/759143601 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c00bab0 con 0x7f1f8c0b14a0 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f8affd700 1 -- 172.21.15.100:0/759143601 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00be70 con 0x7f1f8c0b14a0 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 -- 172.21.15.100:0/759143601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b14a0 msgr2=0x7f1f8c0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 --2- 172.21.15.100:0/759143601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b14a0 0x7f1f8c0cae50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009e60 tx=0x7f1f7c009780 comp rx=0 tx=0).stop 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 -- 172.21.15.100:0/759143601 shutdown_connections 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 --2- 172.21.15.100:0/759143601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b14a0 0x7f1f8c0cae50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 --2- 172.21.15.100:0/759143601 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c0cb390 0x7f1f8c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 -- 172.21.15.100:0/759143601 >> 172.21.15.100:0/759143601 conn(0x7f1f8c0678d0 msgr2=0x7f1f8c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 -- 172.21.15.100:0/759143601 shutdown_connections 2024-08-17T02:22:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 -- 172.21.15.100:0/759143601 wait complete. 2024-08-17T02:22:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 Processor -- start 2024-08-17T02:22:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.125+0000 7f1f917c2700 1 -- start start 2024-08-17T02:22:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f917c2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c0b14a0 0x7f1f8c142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f917c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0cb390 0x7f1f8c069f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f917c2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c154900 0x7f1f8c06a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f917c2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0a6420 con 0x7f1f8c154900 2024-08-17T02:22:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f917c2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c169020 con 0x7f1f8c0b14a0 2024-08-17T02:22:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f917c2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0a6140 con 0x7f1f8c0cb390 2024-08-17T02:22:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c154900 0x7f1f8c06a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c154900 0x7f1f8c06a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36924/0 (socket says 172.21.15.100:36924) 2024-08-17T02:22:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 -- 172.21.15.100:0/1666927681 learned_addr learned my addr 172.21.15.100:0/1666927681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f8b7fe700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0cb390 0x7f1f8c069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f8bfff700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c0b14a0 0x7f1f8c142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 -- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0cb390 msgr2=0x7f1f8c069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0cb390 0x7f1f8c069f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 -- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c0b14a0 msgr2=0x7f1f8c142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c0b14a0 0x7f1f8c142290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 -- 172.21.15.100:0/1666927681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f84009420 con 0x7f1f8c154900 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f8b7fe700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0cb390 0x7f1f8c069f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f90fc1700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c154900 0x7f1f8c06a450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400ea20 tx=0x7f1f840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f83fff700 1 -- 172.21.15.100:0/1666927681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400ee60 con 0x7f1f8c154900 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f83fff700 1 -- 172.21.15.100:0/1666927681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8400c850 con 0x7f1f8c154900 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f83fff700 1 -- 172.21.15.100:0/1666927681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400cc10 con 0x7f1f8c154900 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c009a20 con 0x7f1f8c154900 2024-08-17T02:22:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.129+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c168cb0 con 0x7f1f8c154900 2024-08-17T02:22:01.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.133+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c168830 con 0x7f1f8c154900 2024-08-17T02:22:01.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.133+0000 7f1f83fff700 1 -- 172.21.15.100:0/1666927681 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f1f84013470 con 0x7f1f8c154900 2024-08-17T02:22:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.133+0000 7f1f83fff700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1f6c060800 0x7f1f6c062cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.133+0000 7f1f83fff700 1 -- 172.21.15.100:0/1666927681 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1f8407c6f0 con 0x7f1f8c154900 2024-08-17T02:22:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.133+0000 7f1f8bfff700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1f6c060800 0x7f1f6c062cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.133+0000 7f1f8bfff700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1f6c060800 0x7f1f6c062cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f74007fd0 tx=0x7f1f74009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.141+0000 7f1f83fff700 1 -- 172.21.15.100:0/1666927681 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1f8c168830 con 0x7f1f8c154900 2024-08-17T02:22:01.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.349+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f8c168830 con 0x7f1f8c154900 2024-08-17T02:22:01.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f83fff700 1 -- 172.21.15.100:0/1666927681 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7f1f84017de0 con 0x7f1f8c154900 2024-08-17T02:22:01.354 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:01.354 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-08-17T02:22:01.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1f6c060800 msgr2=0x7f1f6c062cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1f6c060800 0x7f1f6c062cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f74007fd0 tx=0x7f1f74009040 comp rx=0 tx=0).stop 2024-08-17T02:22:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c154900 msgr2=0x7f1f8c06a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c154900 0x7f1f8c06a450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400ea20 tx=0x7f1f840091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 shutdown_connections 2024-08-17T02:22:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1f6c060800 0x7f1f6c062cb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c154900 0x7f1f8c06a450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0cb390 0x7f1f8c069f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 --2- 172.21.15.100:0/1666927681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c0b14a0 0x7f1f8c142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 >> 172.21.15.100:0/1666927681 conn(0x7f1f8c0678d0 msgr2=0x7f1f8c06e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:01.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 shutdown_connections 2024-08-17T02:22:01.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.353+0000 7f1f917c2700 1 -- 172.21.15.100:0/1666927681 wait complete. 2024-08-17T02:22:01.360 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-08-17T02:22:01.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:01.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.461+0000 7f9d87fff700 1 Processor -- start 2024-08-17T02:22:01.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.461+0000 7f9d87fff700 1 -- start start 2024-08-17T02:22:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d881536f0 0x7f9d8815a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d88153b70 con 0x7f9d880cd1b0 2024-08-17T02:22:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8815dbd0 con 0x7f9d88153320 2024-08-17T02:22:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880ce430 con 0x7f9d881536f0 2024-08-17T02:22:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d867fc700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9d880cd1b0 0x7f9d880ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57972/0 (socket says 172.21.15.100:57972) 2024-08-17T02:22:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d867fc700 1 -- 172.21.15.100:0/2325387114 learned_addr learned my addr 172.21.15.100:0/2325387114 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d877fe700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d881536f0 0x7f9d8815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428115390 0 0) 0x7f9d7c001110 con 0x7f9d880cd1b0 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d70003570 con 0x7f9d880cd1b0 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808422373 0 0) 0x7f9d8815dbd0 con 0x7f9d88153320 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d7c001110 con 0x7f9d88153320 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679579552 0 0) 0x7f9d70003570 con 0x7f9d880cd1b0 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d881536f0 msgr2=0x7f9d8815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d881536f0 0x7f9d8815a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 >> v1:172.21.15.123:6789/0 conn(0x7f9d88153320 legacy=0x7f9d880ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8815d370 con 0x7f9d880cd1b0 2024-08-17T02:22:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f9d7c001600 con 0x7f9d880cd1b0 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 >> v1:172.21.15.100:6789/0 conn(0x7f9d880cd1b0 legacy=0x7f9d880ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70003c20 0x7f9d70004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d881536f0 0x7f9d70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70004d50 0x7f9d700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880ce430 con 0x7f9d70003c20 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7000bb70 con 0x7f9d70004d50 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7000bd30 con 0x7f9d881536f0 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d86ffd700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70003c20 0x7f9d70004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d867fc700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d881536f0 0x7f9d70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d877fe700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70004d50 0x7f9d700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d86ffd700 1 -- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d881536f0 msgr2=0x7f9d70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d86ffd700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d881536f0 0x7f9d70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d86ffd700 1 -- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70004d50 msgr2=0x7f9d700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d86ffd700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70004d50 0x7f9d700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d86ffd700 1 -- 172.21.15.100:0/2325387114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70003570 con 0x7f9d70003c20 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d877fe700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70004d50 0x7f9d700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d86ffd700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70003c20 0x7f9d70004030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d80002120 tx=0x7f9d800017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8e54d5f48f7079 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80001ab0 con 0x7f9d70003c20 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8000c980 con 0x7f9d70003c20 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d85ffb700 1 -- 172.21.15.100:0/2325387114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000cd20 con 0x7f9d70003c20 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 -- 172.21.15.100:0/2325387114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70003c20 msgr2=0x7f9d70004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 --2- 172.21.15.100:0/2325387114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70003c20 0x7f9d70004030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d80002120 tx=0x7f9d800017a0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d877fe700 1 -- 172.21.15.100:0/2325387114 reap_dead start 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 -- 172.21.15.100:0/2325387114 shutdown_connections 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 -- 172.21.15.100:0/2325387114 >> 172.21.15.100:0/2325387114 conn(0x7f9d880d00a0 msgr2=0x7f9d880d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 -- 172.21.15.100:0/2325387114 shutdown_connections 2024-08-17T02:22:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.465+0000 7f9d87fff700 1 -- 172.21.15.100:0/2325387114 wait complete. 2024-08-17T02:22:01.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d87fff700 1 Processor -- start 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d87fff700 1 -- start start 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d87fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d70003c20 0x7f9d880b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d87fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d88151080 0x7f9d881514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d87fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880b5220 0x7f9d880b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d87fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d88151a30 con 0x7f9d88151080 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d87fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8815ac20 con 0x7f9d880b5220 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d87fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880b3770 con 0x7f9d70003c20 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d867fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d88151080 0x7f9d881514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d867fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d88151080 0x7f9d881514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36946/0 (socket says 172.21.15.100:36946) 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d867fc700 1 -- 172.21.15.100:0/1391845263 learned_addr learned my addr 172.21.15.100:0/1391845263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:01.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d86ffd700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d70003c20 0x7f9d880b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d877fe700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880b5220 0x7f9d880b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d867fc700 1 -- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d70003c20 msgr2=0x7f9d880b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d867fc700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d70003c20 0x7f9d880b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d867fc700 1 -- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880b5220 msgr2=0x7f9d880b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d867fc700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880b5220 0x7f9d880b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.469+0000 7f9d867fc700 1 -- 172.21.15.100:0/1391845263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8815a310 con 0x7f9d88151080 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d867fc700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d88151080 0x7f9d881514f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00ebc0 tx=0x7f9d8815d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d6effd700 1 -- 172.21.15.100:0/1391845263 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c00c760 con 0x7f9d88151080 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d6effd700 1 -- 172.21.15.100:0/1391845263 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c00c8c0 con 0x7f9d88151080 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d6effd700 1 -- 172.21.15.100:0/1391845263 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c00cc60 con 0x7f9d88151080 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8815a5d0 con 0x7f9d88151080 2024-08-17T02:22:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d880b5bb0 con 0x7f9d88151080 2024-08-17T02:22:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d880638f0 con 0x7f9d88151080 2024-08-17T02:22:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d6effd700 1 -- 172.21.15.100:0/1391845263 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f9d7c01d460 con 0x7f9d88151080 2024-08-17T02:22:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d6effd700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d740607d0 0x7f9d74062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d6effd700 1 -- 172.21.15.100:0/1391845263 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9d7c07c510 con 0x7f9d88151080 2024-08-17T02:22:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.473+0000 7f9d86ffd700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d740607d0 0x7f9d74062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.477+0000 7f9d86ffd700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d740607d0 0x7f9d74062c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d880c9ae0 tx=0x7f9d80016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.481+0000 7f9d6effd700 1 -- 172.21.15.100:0/1391845263 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9d7c04cdc0 con 0x7f9d88151080 2024-08-17T02:22:01.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.645+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d88153b70 con 0x7f9d88151080 2024-08-17T02:22:01.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.645+0000 7f9d6effd700 1 -- 172.21.15.100:0/1391845263 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12188 (secure 0 0 0) 0x7f9d88153b70 con 0x7f9d88151080 2024-08-17T02:22:01.648 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:01.648 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:00.923893+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d740607d0 msgr2=0x7f9d74062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d740607d0 0x7f9d74062c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d880c9ae0 tx=0x7f9d80016000 comp rx=0 tx=0).stop 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d88151080 msgr2=0x7f9d881514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d88151080 0x7f9d881514f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00ebc0 tx=0x7f9d8815d500 comp rx=0 tx=0).stop 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 shutdown_connections 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d740607d0 0x7f9d74062c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880b5220 0x7f9d880b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d88151080 0x7f9d881514f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 --2- 172.21.15.100:0/1391845263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d70003c20 0x7f9d880b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 >> 172.21.15.100:0/1391845263 conn(0x7f9d880d00a0 msgr2=0x7f9d880d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:01.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 shutdown_connections 2024-08-17T02:22:01.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.649+0000 7f9d87fff700 1 -- 172.21.15.100:0/1391845263 wait complete. 2024-08-17T02:22:01.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-17T02:22:01.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:01.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.757+0000 7fbe20aeb700 1 Processor -- start 2024-08-17T02:22:01.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.757+0000 7fbe20aeb700 1 -- start start 2024-08-17T02:22:01.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe1c067030 0x7fbe1c067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c158cc0 0x7fbe1c1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1c0cf670 con 0x7fbe1c0cfb00 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1c0cea60 con 0x7fbe1c067030 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0ced40 con 0x7fbe1c158cc0 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1b7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe1c067030 0x7fbe1c067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe13fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbe1c0cfb00 0x7fbe1c0cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57986/0 (socket says 172.21.15.100:57986) 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe13fff700 1 -- 172.21.15.100:0/1136038963 learned_addr learned my addr 172.21.15.100:0/1136038963 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1bfff700 1 --2- 172.21.15.100:0/1136038963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c158cc0 0x7fbe1c1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1b7fe700 1 -- 172.21.15.100:0/1136038963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c158cc0 msgr2=0x7fbe1c1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1b7fe700 1 --2- 172.21.15.100:0/1136038963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c158cc0 0x7fbe1c1590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1affd700 1 -- 172.21.15.100:0/1136038963 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350228953 0 0) 0x7fbe1c0cf670 con 0x7fbe1c0cfb00 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1b7fe700 1 -- 172.21.15.100:0/1136038963 >> v1:172.21.15.100:6789/0 conn(0x7fbe1c0cfb00 legacy=0x7fbe1c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1b7fe700 1 -- 172.21.15.100:0/1136038963 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c0cf020 con 0x7fbe1c067030 2024-08-17T02:22:01.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1b7fe700 1 --2- 172.21.15.100:0/1136038963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c067030 0x7fbe1c067400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0098e0 tx=0x7fbe0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f28c92940bb78a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1affd700 1 -- 172.21.15.100:0/1136038963 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c009e80 con 0x7fbe1c067030 2024-08-17T02:22:01.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1affd700 1 -- 172.21.15.100:0/1136038963 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c00b860 con 0x7fbe1c067030 2024-08-17T02:22:01.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe1affd700 1 -- 172.21.15.100:0/1136038963 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c00bbe0 con 0x7fbe1c067030 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 -- 172.21.15.100:0/1136038963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c067030 msgr2=0x7fbe1c067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/1136038963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c067030 0x7fbe1c067400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0098e0 tx=0x7fbe0c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 -- 172.21.15.100:0/1136038963 shutdown_connections 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/1136038963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c158cc0 0x7fbe1c1590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/1136038963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c067030 0x7fbe1c067400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 -- 172.21.15.100:0/1136038963 >> 172.21.15.100:0/1136038963 conn(0x7fbe1c0d0b50 msgr2=0x7fbe1c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 -- 172.21.15.100:0/1136038963 shutdown_connections 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.761+0000 7fbe20aeb700 1 -- 172.21.15.100:0/1136038963 wait complete. 2024-08-17T02:22:01.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 Processor -- start 2024-08-17T02:22:01.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 -- start start 2024-08-17T02:22:01.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c0cfb00 0x7fbe1c1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe1c158cc0 0x7fbe1c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c1544d0 0x7fbe1c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c157cd0 con 0x7fbe1c1544d0 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c159fc0 con 0x7fbe1c158cc0 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c1590d0 con 0x7fbe1c0cfb00 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c1544d0 0x7fbe1c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c1544d0 0x7fbe1c15a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36964/0 (socket says 172.21.15.100:36964) 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 -- 172.21.15.100:0/2475008815 learned_addr learned my addr 172.21.15.100:0/2475008815 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1b7fe700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c0cfb00 0x7fbe1c1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe13fff700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe1c158cc0 0x7fbe1c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 -- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c0cfb00 msgr2=0x7fbe1c1536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c0cfb00 0x7fbe1c1536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 -- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe1c158cc0 msgr2=0x7fbe1c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe1c158cc0 0x7fbe1c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 -- 172.21.15.100:0/2475008815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c009420 con 0x7fbe1c1544d0 2024-08-17T02:22:01.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe1bfff700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c1544d0 0x7fbe1c15a450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe14007f20 tx=0x7fbe14009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe137fe700 1 -- 172.21.15.100:0/2475008815 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400eb90 con 0x7fbe1c1544d0 2024-08-17T02:22:01.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe137fe700 1 -- 172.21.15.100:0/2475008815 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1400ed10 con 0x7fbe1c1544d0 2024-08-17T02:22:01.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe137fe700 1 -- 172.21.15.100:0/2475008815 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14014670 con 0x7fbe1c1544d0 2024-08-17T02:22:01.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c1593b0 con 0x7fbe1c1544d0 2024-08-17T02:22:01.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.765+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c159970 con 0x7fbe1c1544d0 2024-08-17T02:22:01.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.769+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c15a150 con 0x7fbe1c1544d0 2024-08-17T02:22:01.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.769+0000 7fbe137fe700 1 -- 172.21.15.100:0/2475008815 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7fbe140149e0 con 0x7fbe1c1544d0 2024-08-17T02:22:01.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.769+0000 7fbe137fe700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbdfc0607d0 0x7fbdfc062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:01.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.769+0000 7fbe137fe700 1 -- 172.21.15.100:0/2475008815 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbe1407cdb0 con 0x7fbe1c1544d0 2024-08-17T02:22:01.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.777+0000 7fbe137fe700 1 -- 172.21.15.100:0/2475008815 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbe1404d630 con 0x7fbe1c1544d0 2024-08-17T02:22:01.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.797+0000 7fbe1b7fe700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbdfc0607d0 0x7fbdfc062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:01.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.797+0000 7fbe1b7fe700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbdfc0607d0 0x7fbdfc062c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c089310 tx=0x7fbe0c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:01.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.941+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe1c0638f0 con 0x7fbe1c1544d0 2024-08-17T02:22:01.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.941+0000 7fbe137fe700 1 -- 172.21.15.100:0/2475008815 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe14050c50 con 0x7fbe1c1544d0 2024-08-17T02:22:01.944 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:01.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbdfc0607d0 msgr2=0x7fbdfc062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbdfc0607d0 0x7fbdfc062c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c089310 tx=0x7fbe0c017000 comp rx=0 tx=0).stop 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c1544d0 msgr2=0x7fbe1c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c1544d0 0x7fbe1c15a450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe14007f20 tx=0x7fbe14009860 comp rx=0 tx=0).stop 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 shutdown_connections 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbdfc0607d0 0x7fbdfc062c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe1c1544d0 0x7fbe1c15a450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe1c158cc0 0x7fbe1c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 --2- 172.21.15.100:0/2475008815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe1c0cfb00 0x7fbe1c1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 >> 172.21.15.100:0/2475008815 conn(0x7fbe1c0d0b50 msgr2=0x7fbe1c0c56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 shutdown_connections 2024-08-17T02:22:01.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:01.945+0000 7fbe20aeb700 1 -- 172.21.15.100:0/2475008815 wait complete. 2024-08-17T02:22:01.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-17T02:22:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa9abb700 1 Processor -- start 2024-08-17T02:22:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa9abb700 1 -- start start 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa9abb700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7effa40688d0 0x7effa4068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa9abb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40aebb0 0x7effa40ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa9abb700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa40697a0 con 0x7effa4154f90 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa9abb700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa40af000 con 0x7effa40688d0 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa9abb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa40abf30 con 0x7effa40aebb0 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa8ab9700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7effa40688d0 0x7effa4068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa8ab9700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7effa40688d0 0x7effa4068ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36978/0 (socket says 172.21.15.100:36978) 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa8ab9700 1 -- 172.21.15.100:0/979281172 learned_addr learned my addr 172.21.15.100:0/979281172 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa92ba700 1 --1- 172.21.15.100:0/979281172 >> v1:172.21.15.100:6789/0 conn(0x7effa4154f90 0x7effa4155410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58002/0 (socket says 172.21.15.100:58002) 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa3fff700 1 --2- 172.21.15.100:0/979281172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40aebb0 0x7effa40ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa37fe700 1 -- 172.21.15.100:0/979281172 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171372795 0 0) 0x7effa40697a0 con 0x7effa4154f90 2024-08-17T02:22:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa37fe700 1 -- 172.21.15.100:0/979281172 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff8c003570 con 0x7effa4154f90 2024-08-17T02:22:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa8ab9700 1 -- 172.21.15.100:0/979281172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40aebb0 msgr2=0x7effa40ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa8ab9700 1 --2- 172.21.15.100:0/979281172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40aebb0 0x7effa40ab9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa8ab9700 1 -- 172.21.15.100:0/979281172 >> v1:172.21.15.100:6789/0 conn(0x7effa4154f90 legacy=0x7effa4155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa8ab9700 1 -- 172.21.15.100:0/979281172 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa40ac1d0 con 0x7effa40688d0 2024-08-17T02:22:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa8ab9700 1 --2- 172.21.15.100:0/979281172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40688d0 0x7effa4068ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff940099b0 tx=0x7eff94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb5351d9c90c748 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa37fe700 1 -- 172.21.15.100:0/979281172 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94009ea0 con 0x7effa40688d0 2024-08-17T02:22:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa37fe700 1 -- 172.21.15.100:0/979281172 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9400b820 con 0x7effa40688d0 2024-08-17T02:22:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.061+0000 7effa37fe700 1 -- 172.21.15.100:0/979281172 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9400bba0 con 0x7effa40688d0 2024-08-17T02:22:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 -- 172.21.15.100:0/979281172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40688d0 msgr2=0x7effa4068ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 --2- 172.21.15.100:0/979281172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40688d0 0x7effa4068ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff940099b0 tx=0x7eff94009250 comp rx=0 tx=0).stop 2024-08-17T02:22:02.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 -- 172.21.15.100:0/979281172 shutdown_connections 2024-08-17T02:22:02.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 --2- 172.21.15.100:0/979281172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40aebb0 0x7effa40ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 --2- 172.21.15.100:0/979281172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40688d0 0x7effa4068ca0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 -- 172.21.15.100:0/979281172 >> 172.21.15.100:0/979281172 conn(0x7effa406d820 msgr2=0x7effa406dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:02.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 -- 172.21.15.100:0/979281172 shutdown_connections 2024-08-17T02:22:02.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 -- 172.21.15.100:0/979281172 wait complete. 2024-08-17T02:22:02.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 Processor -- start 2024-08-17T02:22:02.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.065+0000 7effa9abb700 1 -- start start 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa9abb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40688d0 0x7effa4154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa9abb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40aebb0 0x7effa4158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa9abb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4154f90 0x7effa41470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa9abb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa4158c90 con 0x7effa40aebb0 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa9abb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa4158f30 con 0x7effa4154f90 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa9abb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa4159210 con 0x7effa40688d0 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40aebb0 0x7effa4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40aebb0 0x7effa4158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36992/0 (socket says 172.21.15.100:36992) 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 -- 172.21.15.100:0/4101941255 learned_addr learned my addr 172.21.15.100:0/4101941255 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:02.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa8ab9700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40688d0 0x7effa4154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa92ba700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4154f90 0x7effa41470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 -- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40688d0 msgr2=0x7effa4154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40688d0 0x7effa4154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 -- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4154f90 msgr2=0x7effa41470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4154f90 0x7effa41470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 -- 172.21.15.100:0/4101941255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff940094f0 con 0x7effa40aebb0 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa3fff700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40aebb0 0x7effa4158750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff9800df50 tx=0x7eff9800f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa0ff9700 1 -- 172.21.15.100:0/4101941255 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98012040 con 0x7effa40aebb0 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa0ff9700 1 -- 172.21.15.100:0/4101941255 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98010c00 con 0x7effa40aebb0 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa4155c50 con 0x7effa40aebb0 2024-08-17T02:22:02.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa0ff9700 1 -- 172.21.15.100:0/4101941255 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98018410 con 0x7effa40aebb0 2024-08-17T02:22:02.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.069+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa4156210 con 0x7effa40aebb0 2024-08-17T02:22:02.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.073+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa40638f0 con 0x7effa40aebb0 2024-08-17T02:22:02.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.073+0000 7effa0ff9700 1 -- 172.21.15.100:0/4101941255 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7eff980187c0 con 0x7effa40aebb0 2024-08-17T02:22:02.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.073+0000 7effa0ff9700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7eff900607d0 0x7eff90062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.073+0000 7effa0ff9700 1 -- 172.21.15.100:0/4101941255 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7eff9807edc0 con 0x7effa40aebb0 2024-08-17T02:22:02.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.073+0000 7effa8ab9700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7eff900607d0 0x7eff90062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.073+0000 7effa8ab9700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7eff900607d0 0x7eff90062c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7eff940099b0 tx=0x7eff94016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.081+0000 7effa0ff9700 1 -- 172.21.15.100:0/4101941255 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7eff9804f640 con 0x7effa40aebb0 2024-08-17T02:22:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.241+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7effa40ae990 con 0x7effa40aebb0 2024-08-17T02:22:02.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.241+0000 7effa0ff9700 1 -- 172.21.15.100:0/4101941255 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7eff98052c60 con 0x7effa40aebb0 2024-08-17T02:22:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:02.243 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-08-17T02:22:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-08-17T02:22:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-08-17T02:22:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:02.244 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-08-17T02:22:02.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7eff900607d0 msgr2=0x7eff90062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7eff900607d0 0x7eff90062c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7eff940099b0 tx=0x7eff94016000 comp rx=0 tx=0).stop 2024-08-17T02:22:02.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40aebb0 msgr2=0x7effa4158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40aebb0 0x7effa4158750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff9800df50 tx=0x7eff9800f040 comp rx=0 tx=0).stop 2024-08-17T02:22:02.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 shutdown_connections 2024-08-17T02:22:02.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7eff900607d0 0x7eff90062c80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4154f90 0x7effa41470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa40aebb0 0x7effa4158750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 --2- 172.21.15.100:0/4101941255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effa40688d0 0x7effa4154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 >> 172.21.15.100:0/4101941255 conn(0x7effa406d820 msgr2=0x7effa40b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:02.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 shutdown_connections 2024-08-17T02:22:02.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.245+0000 7effa9abb700 1 -- 172.21.15.100:0/4101941255 wait complete. 2024-08-17T02:22:02.262 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-17T02:22:02.263 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-17T02:22:02.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.373+0000 7f35ada8a700 1 Processor -- start 2024-08-17T02:22:02.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.373+0000 7f35ada8a700 1 -- start start 2024-08-17T02:22:02.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f35a80b2e70 0x7f35a80cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a80c5bb0 con 0x7f35a80cc230 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a80b32f0 con 0x7f35a80ce640 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a80c4fa0 con 0x7f35a80b2e70 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a7fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f35a80b2e70 0x7f35a80cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a7fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f35a80b2e70 0x7f35a80cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36994/0 (socket says 172.21.15.100:36994) 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a7fff700 1 -- 172.21.15.100:0/3714142065 learned_addr learned my addr 172.21.15.100:0/3714142065 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ad289700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f35a80cc230 0x7f35a80c6040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58006/0 (socket says 172.21.15.100:58006) 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a7fff700 1 -- 172.21.15.100:0/3714142065 >> v1:172.21.15.123:6789/0 conn(0x7f35a80ce640 legacy=0x7f35a80cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a77fe700 1 -- 172.21.15.100:0/3714142065 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036909589 0 0) 0x7f35a80c5bb0 con 0x7f35a80cc230 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a7fff700 1 -- 172.21.15.100:0/3714142065 >> v1:172.21.15.100:6789/0 conn(0x7f35a80cc230 legacy=0x7f35a80c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a7fff700 1 -- 172.21.15.100:0/3714142065 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80c5280 con 0x7f35a80b2e70 2024-08-17T02:22:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a7fff700 1 --2- 172.21.15.100:0/3714142065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80b2e70 0x7f35a80cbcf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35980099b0 tx=0x7f3598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac343b0c818a576 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:02.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a77fe700 1 -- 172.21.15.100:0/3714142065 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800b860 con 0x7f35a80b2e70 2024-08-17T02:22:02.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a77fe700 1 -- 172.21.15.100:0/3714142065 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359800b9c0 con 0x7f35a80b2e70 2024-08-17T02:22:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35a77fe700 1 -- 172.21.15.100:0/3714142065 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800bd80 con 0x7f35a80b2e70 2024-08-17T02:22:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 -- 172.21.15.100:0/3714142065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80b2e70 msgr2=0x7f35a80cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 --2- 172.21.15.100:0/3714142065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80b2e70 0x7f35a80cbcf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35980099b0 tx=0x7f3598009250 comp rx=0 tx=0).stop 2024-08-17T02:22:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 -- 172.21.15.100:0/3714142065 shutdown_connections 2024-08-17T02:22:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 --2- 172.21.15.100:0/3714142065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80b2e70 0x7f35a80cbcf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 -- 172.21.15.100:0/3714142065 >> 172.21.15.100:0/3714142065 conn(0x7f35a8099410 msgr2=0x7f35a80675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 -- 172.21.15.100:0/3714142065 shutdown_connections 2024-08-17T02:22:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.377+0000 7f35ada8a700 1 -- 172.21.15.100:0/3714142065 wait complete. 2024-08-17T02:22:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.381+0000 7f35ada8a700 1 Processor -- start 2024-08-17T02:22:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.381+0000 7f35ada8a700 1 -- start start 2024-08-17T02:22:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.381+0000 7f35ada8a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35a80b2e70 0x7f35a809aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ada8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a80cc230 0x7f35a809bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ada8a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80ce640 0x7f35a80c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ada8a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a809aa30 con 0x7f35a80ce640 2024-08-17T02:22:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ada8a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a8099e20 con 0x7f35a80cc230 2024-08-17T02:22:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ada8a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a809a100 con 0x7f35a80b2e70 2024-08-17T02:22:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80ce640 0x7f35a80c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80ce640 0x7f35a80c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37008/0 (socket says 172.21.15.100:37008) 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 -- 172.21.15.100:0/935519216 learned_addr learned my addr 172.21.15.100:0/935519216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35aca88700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35a80b2e70 0x7f35a809aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35a7fff700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a80cc230 0x7f35a809bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 -- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35a80b2e70 msgr2=0x7f35a809aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35a80b2e70 0x7f35a809aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 -- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a80cc230 msgr2=0x7f35a809bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a80cc230 0x7f35a809bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 -- 172.21.15.100:0/935519216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35980094f0 con 0x7f35a80ce640 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35a7fff700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a80cc230 0x7f35a809bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ad289700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80ce640 0x7f35a80c96c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35a000cd00 tx=0x7f35a0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35a4ff9700 1 -- 172.21.15.100:0/935519216 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000ce60 con 0x7f35a80ce640 2024-08-17T02:22:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35a4ff9700 1 -- 172.21.15.100:0/935519216 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a0005df0 con 0x7f35a80ce640 2024-08-17T02:22:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a809a3e0 con 0x7f35a80ce640 2024-08-17T02:22:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35a4ff9700 1 -- 172.21.15.100:0/935519216 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000ab90 con 0x7f35a80ce640 2024-08-17T02:22:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a809b400 con 0x7f35a80ce640 2024-08-17T02:22:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.385+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a80644b0 con 0x7f35a80ce640 2024-08-17T02:22:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.389+0000 7f35a4ff9700 1 -- 172.21.15.100:0/935519216 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f35a00070f0 con 0x7f35a80ce640 2024-08-17T02:22:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.389+0000 7f35a4ff9700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3588060850 0x7f3588062d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.389+0000 7f35a4ff9700 1 -- 172.21.15.100:0/935519216 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f35a007c900 con 0x7f35a80ce640 2024-08-17T02:22:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.389+0000 7f35aca88700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3588060850 0x7f3588062d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:02.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.389+0000 7f35aca88700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3588060850 0x7f3588062d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f35a80b5a80 tx=0x7f3590009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:02.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.393+0000 7f35a4ff9700 1 -- 172.21.15.100:0/935519216 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f35a004d160 con 0x7f35a80ce640 2024-08-17T02:22:02.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:02.557+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f35a80ced90 con 0x7f35a80ce640 2024-08-17T02:22:03.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.089+0000 7f35a4ff9700 1 -- 172.21.15.100:0/935519216 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f35a0050780 con 0x7f35a80ce640 2024-08-17T02:22:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.097+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3588060850 msgr2=0x7f3588062d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.097+0000 7f35ada8a700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3588060850 0x7f3588062d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f35a80b5a80 tx=0x7f3590009040 comp rx=0 tx=0).stop 2024-08-17T02:22:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.097+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80ce640 msgr2=0x7f35a80c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.097+0000 7f35ada8a700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80ce640 0x7f35a80c96c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35a000cd00 tx=0x7f35a0001ac0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.101+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 shutdown_connections 2024-08-17T02:22:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.101+0000 7f35ada8a700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3588060850 0x7f3588062d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.101+0000 7f35ada8a700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80ce640 0x7f35a80c96c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.101+0000 7f35ada8a700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a80cc230 0x7f35a809bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.101+0000 7f35ada8a700 1 --2- 172.21.15.100:0/935519216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35a80b2e70 0x7f35a809aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.101+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 >> 172.21.15.100:0/935519216 conn(0x7f35a8099410 msgr2=0x7f35a8158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.101+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 shutdown_connections 2024-08-17T02:22:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.101+0000 7f35ada8a700 1 -- 172.21.15.100:0/935519216 wait complete. 2024-08-17T02:22:03.105 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-08-17T02:22:03.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-17T02:22:03.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.233+0000 7f2290fb5700 1 Processor -- start 2024-08-17T02:22:03.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.233+0000 7f2290fb5700 1 -- start start 2024-08-17T02:22:03.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f2290fb5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0b1960 0x7f228c0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:03.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f2290fb5700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c0b0600 con 0x7f228c0cd480 2024-08-17T02:22:03.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f2290fb5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c068960 con 0x7f228c0cd0b0 2024-08-17T02:22:03.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f2290fb5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0afd60 con 0x7f228c0b1960 2024-08-17T02:22:03.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f2282ffd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f228c0cd480 0x7f228c0b1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58012/0 (socket says 172.21.15.100:58012) 2024-08-17T02:22:03.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f2282ffd700 1 -- 172.21.15.100:0/3242646818 learned_addr learned my addr 172.21.15.100:0/3242646818 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:03.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228bfff700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0b1960 0x7f228c0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:03.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714895949 0 0) 0x7f228c0b0600 con 0x7f228c0cd480 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f226c003570 con 0x7f228c0cd480 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3829952938 0 0) 0x7f226c003570 con 0x7f228c0cd480 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0b1960 msgr2=0x7f228c0b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0b1960 0x7f228c0b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 >> v1:172.21.15.123:6789/0 conn(0x7f228c0cd0b0 legacy=0x7f228c0af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c0b0040 con 0x7f228c0cd480 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f2274001470 con 0x7f228c0cd480 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 >> v1:172.21.15.100:6789/0 conn(0x7f228c0cd480 legacy=0x7f228c0b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:03.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f226c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0b1960 0x7f226c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226c004d50 0x7f226c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0afd60 con 0x7f226c003a90 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c00bc20 con 0x7f226c004d50 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c00bde0 con 0x7f228c0b1960 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f2282ffd700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0b1960 0x7f226c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228b7fe700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f226c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.237+0000 7f228bfff700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226c004d50 0x7f226c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228b7fe700 1 -- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0b1960 msgr2=0x7f226c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228b7fe700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0b1960 0x7f226c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228b7fe700 1 -- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226c004d50 msgr2=0x7f226c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228b7fe700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226c004d50 0x7f226c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228b7fe700 1 -- 172.21.15.100:0/3242646818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c003570 con 0x7f226c003a90 2024-08-17T02:22:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228b7fe700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f226c003ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f227c001880 tx=0x7f227c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7349efa8a9043eb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:03.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c012420 con 0x7f226c003a90 2024-08-17T02:22:03.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c012580 con 0x7f226c003a90 2024-08-17T02:22:03.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228affd700 1 -- 172.21.15.100:0/3242646818 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c012920 con 0x7f226c003a90 2024-08-17T02:22:03.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f2290fb5700 1 -- 172.21.15.100:0/3242646818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 msgr2=0x7f226c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:03.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f2290fb5700 1 --2- 172.21.15.100:0/3242646818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f226c003ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f227c001880 tx=0x7f227c007d30 comp rx=0 tx=0).stop 2024-08-17T02:22:03.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f228bfff700 1 -- 172.21.15.100:0/3242646818 reap_dead start 2024-08-17T02:22:03.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f2290fb5700 1 -- 172.21.15.100:0/3242646818 shutdown_connections 2024-08-17T02:22:03.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f2290fb5700 1 -- 172.21.15.100:0/3242646818 >> 172.21.15.100:0/3242646818 conn(0x7f228c0ce120 msgr2=0x7f228c0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:03.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f2290fb5700 1 -- 172.21.15.100:0/3242646818 shutdown_connections 2024-08-17T02:22:03.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.241+0000 7f2290fb5700 1 -- 172.21.15.100:0/3242646818 wait complete. 2024-08-17T02:22:03.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2290fb5700 1 Processor -- start 2024-08-17T02:22:03.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2290fb5700 1 -- start start 2024-08-17T02:22:03.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2290fb5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f228c14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:03.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2290fb5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c0a5e30 0x7f228c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:03.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2290fb5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0c9760 0x7f228c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2290fb5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0c72d0 con 0x7f226c003a90 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2290fb5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0c7850 con 0x7f228c0a5e30 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2290fb5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0ca0f0 con 0x7f228c0c9760 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f228c14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f228c14aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37034/0 (socket says 172.21.15.100:37034) 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 -- 172.21.15.100:0/790125226 learned_addr learned my addr 172.21.15.100:0/790125226 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228bfff700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0c9760 0x7f228c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f2282ffd700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c0a5e30 0x7f228c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 -- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0c9760 msgr2=0x7f228c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0c9760 0x7f228c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 -- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c0a5e30 msgr2=0x7f228c0a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c0a5e30 0x7f228c0a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 -- 172.21.15.100:0/790125226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c0a68b0 con 0x7f226c003a90 2024-08-17T02:22:03.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.245+0000 7f228b7fe700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f228c14aac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f227c001e70 tx=0x7f227c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:03.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2283fff700 1 -- 172.21.15.100:0/790125226 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c012420 con 0x7f226c003a90 2024-08-17T02:22:03.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2283fff700 1 -- 172.21.15.100:0/790125226 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c001200 con 0x7f226c003a90 2024-08-17T02:22:03.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c0a6b90 con 0x7f226c003a90 2024-08-17T02:22:03.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2283fff700 1 -- 172.21.15.100:0/790125226 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c0126f0 con 0x7f226c003a90 2024-08-17T02:22:03.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c16a8e0 con 0x7f226c003a90 2024-08-17T02:22:03.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2283fff700 1 -- 172.21.15.100:0/790125226 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f227c019030 con 0x7f226c003a90 2024-08-17T02:22:03.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c0638f0 con 0x7f226c003a90 2024-08-17T02:22:03.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2283fff700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f22700607d0 0x7f2270062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:03.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2283fff700 1 -- 172.21.15.100:0/790125226 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f227c07dc00 con 0x7f226c003a90 2024-08-17T02:22:03.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.249+0000 7f2282ffd700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f22700607d0 0x7f2270062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:03.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.253+0000 7f2282ffd700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f22700607d0 0x7f2270062c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22740015a0 tx=0x7f2274009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:03.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.257+0000 7f2283fff700 1 -- 172.21.15.100:0/790125226 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f227c04e3c0 con 0x7f226c003a90 2024-08-17T02:22:03.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:03.417+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f228c0636b0 con 0x7f226c003a90 2024-08-17T02:22:04.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.109+0000 7f2283fff700 1 -- 172.21.15.100:0/790125226 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f227c04e790 con 0x7f226c003a90 2024-08-17T02:22:04.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.109+0000 7f2283fff700 1 -- 172.21.15.100:0/790125226 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f227c0677e0 con 0x7f226c003a90 2024-08-17T02:22:04.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f22700607d0 msgr2=0x7f2270062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:04.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f22700607d0 0x7f2270062c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22740015a0 tx=0x7f2274009040 comp rx=0 tx=0).stop 2024-08-17T02:22:04.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 msgr2=0x7f228c14aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:04.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f228c14aac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f227c001e70 tx=0x7f227c015000 comp rx=0 tx=0).stop 2024-08-17T02:22:04.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 shutdown_connections 2024-08-17T02:22:04.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f22700607d0 0x7f2270062c80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f228c0c9760 0x7f228c0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c0a5e30 0x7f228c0a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 --2- 172.21.15.100:0/790125226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c003a90 0x7f228c14aac0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 >> 172.21.15.100:0/790125226 conn(0x7f228c0ce120 msgr2=0x7f228c158c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:04.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.117+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 shutdown_connections 2024-08-17T02:22:04.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.121+0000 7f2290fb5700 1 -- 172.21.15.100:0/790125226 wait complete. 2024-08-17T02:22:04.127 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-08-17T02:22:04.143 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-17T02:22:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.241+0000 7f47b759e700 1 Processor -- start 2024-08-17T02:22:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.241+0000 7f47b759e700 1 -- start start 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b8067510 0x7f47b80ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47b8159480 0x7f47b80cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b80ce210 con 0x7f47b81590b0 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b80cba00 con 0x7f47b8159480 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80cdc90 con 0x7f47b8067510 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b659c700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f47b81590b0 0x7f47b8066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58022/0 (socket says 172.21.15.100:58022) 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b659c700 1 -- 172.21.15.100:0/1986521880 learned_addr learned my addr 172.21.15.100:0/1986521880 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47add9b700 1 --2- 172.21.15.100:0/1986521880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b8067510 0x7f47b80ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b6d9d700 1 --2- 172.21.15.100:0/1986521880 >> v2:172.21.15.123:3300/0 conn(0x7f47b8159480 0x7f47b80cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b5d9b700 1 -- 172.21.15.100:0/1986521880 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139195563 0 0) 0x7f47b80ce210 con 0x7f47b81590b0 2024-08-17T02:22:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b5d9b700 1 -- 172.21.15.100:0/1986521880 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a4003570 con 0x7f47b81590b0 2024-08-17T02:22:04.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47add9b700 1 -- 172.21.15.100:0/1986521880 >> v2:172.21.15.123:3300/0 conn(0x7f47b8159480 msgr2=0x7f47b80cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:04.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47add9b700 1 --2- 172.21.15.100:0/1986521880 >> v2:172.21.15.123:3300/0 conn(0x7f47b8159480 0x7f47b80cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47add9b700 1 -- 172.21.15.100:0/1986521880 >> v1:172.21.15.100:6789/0 conn(0x7f47b81590b0 legacy=0x7f47b8066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:04.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b5d9b700 1 -- 172.21.15.100:0/1986521880 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 820159251 0 0) 0x7f47a4003570 con 0x7f47b81590b0 2024-08-17T02:22:04.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47add9b700 1 -- 172.21.15.100:0/1986521880 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b80cb440 con 0x7f47b8067510 2024-08-17T02:22:04.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47add9b700 1 --2- 172.21.15.100:0/1986521880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b8067510 0x7f47b80ce6a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f47a00098e0 tx=0x7f47a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ea0e4c4efab8cd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:04.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b5d9b700 1 -- 172.21.15.100:0/1986521880 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8001470 con 0x7f47b8067510 2024-08-17T02:22:04.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b5d9b700 1 -- 172.21.15.100:0/1986521880 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0011420 con 0x7f47b8067510 2024-08-17T02:22:04.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b5d9b700 1 -- 172.21.15.100:0/1986521880 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a00117c0 con 0x7f47b8067510 2024-08-17T02:22:04.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 -- 172.21.15.100:0/1986521880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b8067510 msgr2=0x7f47b80ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 --2- 172.21.15.100:0/1986521880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b8067510 0x7f47b80ce6a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f47a00098e0 tx=0x7f47a00091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 -- 172.21.15.100:0/1986521880 shutdown_connections 2024-08-17T02:22:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 --2- 172.21.15.100:0/1986521880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b8067510 0x7f47b80ce6a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 --2- 172.21.15.100:0/1986521880 >> v2:172.21.15.123:3300/0 conn(0x7f47b8159480 0x7f47b80cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 -- 172.21.15.100:0/1986521880 >> 172.21.15.100:0/1986521880 conn(0x7f47b80d0b90 msgr2=0x7f47b80ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.245+0000 7f47b759e700 1 -- 172.21.15.100:0/1986521880 shutdown_connections 2024-08-17T02:22:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 -- 172.21.15.100:0/1986521880 wait complete. 2024-08-17T02:22:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 Processor -- start 2024-08-17T02:22:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 -- start start 2024-08-17T02:22:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8067510 0x7f47b81542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b81590b0 0x7f47b8157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b8159480 0x7f47b80b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b815ee70 con 0x7f47b8159480 2024-08-17T02:22:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b815e1f0 con 0x7f47b8067510 2024-08-17T02:22:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b815e4d0 con 0x7f47b81590b0 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b8159480 0x7f47b80b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8067510 0x7f47b81542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b8159480 0x7f47b80b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37042/0 (socket says 172.21.15.100:37042) 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47add9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b81590b0 0x7f47b8157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.249+0000 7f47b6d9d700 1 -- 172.21.15.100:0/4070868534 learned_addr learned my addr 172.21.15.100:0/4070868534 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b6d9d700 1 -- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b81590b0 msgr2=0x7f47b8157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b6d9d700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b81590b0 0x7f47b8157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b6d9d700 1 -- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8067510 msgr2=0x7f47b81542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b6d9d700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8067510 0x7f47b81542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b6d9d700 1 -- 172.21.15.100:0/4070868534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a0009420 con 0x7f47b8159480 2024-08-17T02:22:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b6d9d700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b8159480 0x7f47b80b3670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009d40 tx=0x7f47b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47aeffd700 1 -- 172.21.15.100:0/4070868534 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0009ea0 con 0x7f47b8159480 2024-08-17T02:22:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47aeffd700 1 -- 172.21.15.100:0/4070868534 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b000f040 con 0x7f47b8159480 2024-08-17T02:22:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b815e7b0 con 0x7f47b8159480 2024-08-17T02:22:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47aeffd700 1 -- 172.21.15.100:0/4070868534 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000ccf0 con 0x7f47b8159480 2024-08-17T02:22:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b815f290 con 0x7f47b8159480 2024-08-17T02:22:04.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.253+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8064330 con 0x7f47b8159480 2024-08-17T02:22:04.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.257+0000 7f47aeffd700 1 -- 172.21.15.100:0/4070868534 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f47b0014690 con 0x7f47b8159480 2024-08-17T02:22:04.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.257+0000 7f47aeffd700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4798060860 0x7f4798062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:04.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.257+0000 7f47aeffd700 1 -- 172.21.15.100:0/4070868534 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f47b007d400 con 0x7f47b8159480 2024-08-17T02:22:04.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.257+0000 7f47b659c700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4798060860 0x7f4798062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:04.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.257+0000 7f47b659c700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4798060860 0x7f4798062d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47a80011e0 tx=0x7f47a8001aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:04.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.265+0000 7f47aeffd700 1 -- 172.21.15.100:0/4070868534 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f47b004d8c0 con 0x7f47b8159480 2024-08-17T02:22:04.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:04.457+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f47b8070cf0 con 0x7f47b8159480 2024-08-17T02:22:05.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.109+0000 7f47aeffd700 1 -- 172.21.15.100:0/4070868534 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f47b0050ee0 con 0x7f47b8159480 2024-08-17T02:22:05.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4798060860 msgr2=0x7f4798062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4798060860 0x7f4798062d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47a80011e0 tx=0x7f47a8001aa0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b8159480 msgr2=0x7f47b80b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b8159480 0x7f47b80b3670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009d40 tx=0x7f47b000b040 comp rx=0 tx=0).stop 2024-08-17T02:22:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 shutdown_connections 2024-08-17T02:22:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4798060860 0x7f4798062d10 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47a80011e0 tx=0x7f47a8001aa0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b8159480 0x7f47b80b3670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47b81590b0 0x7f47b8157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 --2- 172.21.15.100:0/4070868534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8067510 0x7f47b81542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 >> 172.21.15.100:0/4070868534 conn(0x7f47b80d0b90 msgr2=0x7f47b8066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.117+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 shutdown_connections 2024-08-17T02:22:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.121+0000 7f47b759e700 1 -- 172.21.15.100:0/4070868534 wait complete. 2024-08-17T02:22:05.128 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 2 and data pool 3 2024-08-17T02:22:05.146 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:05.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 Processor -- start 2024-08-17T02:22:05.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- start start 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f451c15d5b0 0x7f451c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451c0cf190 con 0x7f451c0d0010 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451c0d09a0 con 0x7f451c15d5b0 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451c0ce5e0 con 0x7f451c159040 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f452267d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f451c15d5b0 0x7f451c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f452267d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f451c15d5b0 0x7f451c0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56612/0 (socket says 172.21.15.100:56612) 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f452267d700 1 -- 172.21.15.100:0/1685928159 learned_addr learned my addr 172.21.15.100:0/1685928159 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4521e7c700 1 --2- 172.21.15.100:0/1685928159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f452267d700 1 -- 172.21.15.100:0/1685928159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 msgr2=0x7f451c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f452267d700 1 --2- 172.21.15.100:0/1685928159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f452267d700 1 -- 172.21.15.100:0/1685928159 >> v1:172.21.15.123:6789/0 conn(0x7f451c0d0010 legacy=0x7f451c15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f452267d700 1 -- 172.21.15.100:0/1685928159 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c0ce860 con 0x7f451c15d5b0 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4521e7c700 1 --2- 172.21.15.100:0/1685928159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f452267d700 1 --2- 172.21.15.100:0/1685928159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c15d5b0 0x7f451c0d0460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45140099b0 tx=0x7f4514009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c839c2b5729723 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4520e7a700 1 -- 172.21.15.100:0/1685928159 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451400b860 con 0x7f451c15d5b0 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4520e7a700 1 -- 172.21.15.100:0/1685928159 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451400b9c0 con 0x7f451c15d5b0 2024-08-17T02:22:05.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4520e7a700 1 -- 172.21.15.100:0/1685928159 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451400bd60 con 0x7f451c15d5b0 2024-08-17T02:22:05.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- 172.21.15.100:0/1685928159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c15d5b0 msgr2=0x7f451c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 --2- 172.21.15.100:0/1685928159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c15d5b0 0x7f451c0d0460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45140099b0 tx=0x7f4514009250 comp rx=0 tx=0).stop 2024-08-17T02:22:05.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- 172.21.15.100:0/1685928159 shutdown_connections 2024-08-17T02:22:05.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 --2- 172.21.15.100:0/1685928159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c15d5b0 0x7f451c0d0460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 --2- 172.21.15.100:0/1685928159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- 172.21.15.100:0/1685928159 >> 172.21.15.100:0/1685928159 conn(0x7f451c068250 msgr2=0x7f451c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:05.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- 172.21.15.100:0/1685928159 shutdown_connections 2024-08-17T02:22:05.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.245+0000 7f4522e7e700 1 -- 172.21.15.100:0/1685928159 wait complete. 2024-08-17T02:22:05.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 Processor -- start 2024-08-17T02:22:05.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 -- start start 2024-08-17T02:22:05.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c0d0010 0x7f451c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c1411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f451c15d5b0 0x7f451c0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c0c5750 con 0x7f451c0d0010 2024-08-17T02:22:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c0c6120 con 0x7f451c15d5b0 2024-08-17T02:22:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451c0c6d90 con 0x7f451c159040 2024-08-17T02:22:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c0d0010 0x7f451c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c0d0010 0x7f451c0c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56614/0 (socket says 172.21.15.100:56614) 2024-08-17T02:22:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 -- 172.21.15.100:0/897953122 learned_addr learned my addr 172.21.15.100:0/897953122 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f452167b700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c1411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f452267d700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f451c15d5b0 0x7f451c0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 -- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 msgr2=0x7f451c1411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c1411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 -- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f451c15d5b0 msgr2=0x7f451c0c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f451c15d5b0 0x7f451c0c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 -- 172.21.15.100:0/897953122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140094f0 con 0x7f451c0d0010 2024-08-17T02:22:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4521e7c700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c0d0010 0x7f451c0c5be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f451800d1d0 tx=0x7f4518009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4511ffb700 1 -- 172.21.15.100:0/897953122 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451800dd10 con 0x7f451c0d0010 2024-08-17T02:22:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4511ffb700 1 -- 172.21.15.100:0/897953122 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451800f040 con 0x7f451c0d0010 2024-08-17T02:22:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.249+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451c0c4e20 con 0x7f451c0d0010 2024-08-17T02:22:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.253+0000 7f4511ffb700 1 -- 172.21.15.100:0/897953122 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518015670 con 0x7f451c0d0010 2024-08-17T02:22:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.253+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451c0c53e0 con 0x7f451c0d0010 2024-08-17T02:22:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.253+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451c0643a0 con 0x7f451c0d0010 2024-08-17T02:22:05.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.253+0000 7f4511ffb700 1 -- 172.21.15.100:0/897953122 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4518015a60 con 0x7f451c0d0010 2024-08-17T02:22:05.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.253+0000 7f4511ffb700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4500060860 0x7f4500062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.253+0000 7f4511ffb700 1 -- 172.21.15.100:0/897953122 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f451807cef0 con 0x7f451c0d0010 2024-08-17T02:22:05.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.253+0000 7f452167b700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4500060860 0x7f4500062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.257+0000 7f452167b700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4500060860 0x7f4500062d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f451c148110 tx=0x7f450c007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.265+0000 7f4511ffb700 1 -- 172.21.15.100:0/897953122 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f451804d3f0 con 0x7f451c0d0010 2024-08-17T02:22:05.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.429+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f451c0b3480 con 0x7f451c0d0010 2024-08-17T02:22:05.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.429+0000 7f4511ffb700 1 -- 172.21.15.100:0/897953122 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f4518050a10 con 0x7f451c0d0010 2024-08-17T02:22:05.431 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:05.431 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":21,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:04.459232+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:05.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4500060860 msgr2=0x7f4500062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4500060860 0x7f4500062d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f451c148110 tx=0x7f450c007ec0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c0d0010 msgr2=0x7f451c0c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c0d0010 0x7f451c0c5be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f451800d1d0 tx=0x7f4518009a90 comp rx=0 tx=0).stop 2024-08-17T02:22:05.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 shutdown_connections 2024-08-17T02:22:05.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4500060860 0x7f4500062d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f451c15d5b0 0x7f451c0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f451c159040 0x7f451c1411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 --2- 172.21.15.100:0/897953122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f451c0d0010 0x7f451c0c5be0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 >> 172.21.15.100:0/897953122 conn(0x7f451c068250 msgr2=0x7f451c153ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:05.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 shutdown_connections 2024-08-17T02:22:05.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.433+0000 7f4522e7e700 1 -- 172.21.15.100:0/897953122 wait complete. 2024-08-17T02:22:05.445 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:05.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f92a90b2700 1 Processor -- start 2024-08-17T02:22:05.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f92a90b2700 1 -- start start 2024-08-17T02:22:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f92a90b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4156f80 0x7f92a40ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f92a90b2700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f92a40ace80 0x7f92a4157350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f92a90b2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40adf50 con 0x7f92a4157890 2024-08-17T02:22:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f92a90b2700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40ae920 con 0x7f92a40ace80 2024-08-17T02:22:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f92a90b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40ad340 con 0x7f92a4156f80 2024-08-17T02:22:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f929bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f92a40ace80 0x7f92a4157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f929bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f92a40ace80 0x7f92a4157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56618/0 (socket says 172.21.15.100:56618) 2024-08-17T02:22:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f929bfff700 1 -- 172.21.15.100:0/3683072900 learned_addr learned my addr 172.21.15.100:0/3683072900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.549+0000 7f92a2d9d700 1 --2- 172.21.15.100:0/3683072900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4156f80 0x7f92a40ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f929bfff700 1 -- 172.21.15.100:0/3683072900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4156f80 msgr2=0x7f92a40ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:05.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f929bfff700 1 --2- 172.21.15.100:0/3683072900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4156f80 0x7f92a40ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f929bfff700 1 -- 172.21.15.100:0/3683072900 >> v1:172.21.15.123:6789/0 conn(0x7f92a4157890 legacy=0x7f92a40aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f929bfff700 1 -- 172.21.15.100:0/3683072900 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40ad620 con 0x7f92a40ace80 2024-08-17T02:22:05.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f929bfff700 1 --2- 172.21.15.100:0/3683072900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a40ace80 0x7f92a4157350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f928c0099b0 tx=0x7f928c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14da821f57c012a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a259c700 1 -- 172.21.15.100:0/3683072900 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c00b860 con 0x7f92a40ace80 2024-08-17T02:22:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a259c700 1 -- 172.21.15.100:0/3683072900 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c00b9c0 con 0x7f92a40ace80 2024-08-17T02:22:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a259c700 1 -- 172.21.15.100:0/3683072900 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c00bd60 con 0x7f92a40ace80 2024-08-17T02:22:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 -- 172.21.15.100:0/3683072900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a40ace80 msgr2=0x7f92a4157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 --2- 172.21.15.100:0/3683072900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a40ace80 0x7f92a4157350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f928c0099b0 tx=0x7f928c009250 comp rx=0 tx=0).stop 2024-08-17T02:22:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 -- 172.21.15.100:0/3683072900 shutdown_connections 2024-08-17T02:22:05.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 --2- 172.21.15.100:0/3683072900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a40ace80 0x7f92a4157350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 --2- 172.21.15.100:0/3683072900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4156f80 0x7f92a40ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 -- 172.21.15.100:0/3683072900 >> 172.21.15.100:0/3683072900 conn(0x7f92a4066e00 msgr2=0x7f92a4067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:05.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 -- 172.21.15.100:0/3683072900 shutdown_connections 2024-08-17T02:22:05.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 -- 172.21.15.100:0/3683072900 wait complete. 2024-08-17T02:22:05.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 Processor -- start 2024-08-17T02:22:05.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 -- start start 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a40ace80 0x7f92a40690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.553+0000 7f92a90b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a4156f80 0x7f92a416d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a90b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4157890 0x7f92a416d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a90b2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4097a10 con 0x7f92a4156f80 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a90b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4069620 con 0x7f92a40ace80 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a90b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4097730 con 0x7f92a4157890 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a4156f80 0x7f92a416d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a4156f80 0x7f92a416d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56632/0 (socket says 172.21.15.100:56632) 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 -- 172.21.15.100:0/4162286233 learned_addr learned my addr 172.21.15.100:0/4162286233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a359e700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4157890 0x7f92a416d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a2d9d700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a40ace80 0x7f92a40690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 -- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4157890 msgr2=0x7f92a416d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4157890 0x7f92a416d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 -- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a40ace80 msgr2=0x7f92a40690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a40ace80 0x7f92a40690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 -- 172.21.15.100:0/4162286233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0094f0 con 0x7f92a4156f80 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929bfff700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a4156f80 0x7f92a416d3b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f928c009290 tx=0x7f928c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929affd700 1 -- 172.21.15.100:0/4162286233 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c011e00 con 0x7f92a4156f80 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929affd700 1 -- 172.21.15.100:0/4162286233 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c024440 con 0x7f92a4156f80 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a40affd0 con 0x7f92a4156f80 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929affd700 1 -- 172.21.15.100:0/4162286233 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c0247e0 con 0x7f92a4156f80 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a40b0590 con 0x7f92a4156f80 2024-08-17T02:22:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a40645c0 con 0x7f92a4156f80 2024-08-17T02:22:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.557+0000 7f929affd700 1 -- 172.21.15.100:0/4162286233 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f928c01a420 con 0x7f92a4156f80 2024-08-17T02:22:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.561+0000 7f929affd700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9284060860 0x7f9284062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.561+0000 7f929affd700 1 -- 172.21.15.100:0/4162286233 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f928c07eae0 con 0x7f92a4156f80 2024-08-17T02:22:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.561+0000 7f92a2d9d700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9284060860 0x7f9284062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.561+0000 7f92a2d9d700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9284060860 0x7f9284062d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9294006fd0 tx=0x7f9294008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.565+0000 7f929affd700 1 -- 172.21.15.100:0/4162286233 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f928c04ef30 con 0x7f92a4156f80 2024-08-17T02:22:05.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.733+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92a4099e80 con 0x7f92a4156f80 2024-08-17T02:22:05.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.733+0000 7f929affd700 1 -- 172.21.15.100:0/4162286233 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f928c052550 con 0x7f92a4156f80 2024-08-17T02:22:05.735 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:05.736 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":21,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:04.459232+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:05.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9284060860 msgr2=0x7f9284062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9284060860 0x7f9284062d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9294006fd0 tx=0x7f9294008040 comp rx=0 tx=0).stop 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a4156f80 msgr2=0x7f92a416d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a4156f80 0x7f92a416d3b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f928c009290 tx=0x7f928c017000 comp rx=0 tx=0).stop 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 shutdown_connections 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9284060860 0x7f9284062d10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a4157890 0x7f92a416d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a4156f80 0x7f92a416d3b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 --2- 172.21.15.100:0/4162286233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a40ace80 0x7f92a40690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 >> 172.21.15.100:0/4162286233 conn(0x7f92a4066e00 msgr2=0x7f92a40aa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 shutdown_connections 2024-08-17T02:22:05.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.737+0000 7f92a90b2700 1 -- 172.21.15.100:0/4162286233 wait complete. 2024-08-17T02:22:05.750 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-17T02:22:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.857+0000 7f8d2bfff700 1 Processor -- start 2024-08-17T02:22:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.857+0000 7f8d2bfff700 1 -- start start 2024-08-17T02:22:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d2c0cfb00 0x7f8d2c067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0cf670 con 0x7f8d2c158cc0 2024-08-17T02:22:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0cea60 con 0x7f8d2c067030 2024-08-17T02:22:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0ced40 con 0x7f8d2c0cfb00 2024-08-17T02:22:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2a7fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d2c0cfb00 0x7f8d2c067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2a7fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d2c0cfb00 0x7f8d2c067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56634/0 (socket says 172.21.15.100:56634) 2024-08-17T02:22:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2a7fc700 1 -- 172.21.15.100:0/2766246170 learned_addr learned my addr 172.21.15.100:0/2766246170 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2a7fc700 1 -- 172.21.15.100:0/2766246170 >> v1:172.21.15.123:6789/0 conn(0x7f8d2c067030 legacy=0x7f8d2c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2a7fc700 1 -- 172.21.15.100:0/2766246170 >> v1:172.21.15.100:6789/0 conn(0x7f8d2c158cc0 legacy=0x7f8d2c1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2a7fc700 1 -- 172.21.15.100:0/2766246170 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0cf020 con 0x7f8d2c0cfb00 2024-08-17T02:22:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2a7fc700 1 --2- 172.21.15.100:0/2766246170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c0cfb00 0x7f8d2c067400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d140099b0 tx=0x7f8d14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4965be9c49396a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d29ffb700 1 -- 172.21.15.100:0/2766246170 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400b860 con 0x7f8d2c0cfb00 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d29ffb700 1 -- 172.21.15.100:0/2766246170 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1400b9c0 con 0x7f8d2c0cfb00 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d29ffb700 1 -- 172.21.15.100:0/2766246170 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400bd60 con 0x7f8d2c0cfb00 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 -- 172.21.15.100:0/2766246170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c0cfb00 msgr2=0x7f8d2c067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 --2- 172.21.15.100:0/2766246170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c0cfb00 0x7f8d2c067400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d140099b0 tx=0x7f8d14009250 comp rx=0 tx=0).stop 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 -- 172.21.15.100:0/2766246170 shutdown_connections 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 --2- 172.21.15.100:0/2766246170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c0cfb00 0x7f8d2c067400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 -- 172.21.15.100:0/2766246170 >> 172.21.15.100:0/2766246170 conn(0x7f8d2c0d0b50 msgr2=0x7f8d2c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 -- 172.21.15.100:0/2766246170 shutdown_connections 2024-08-17T02:22:05.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.861+0000 7f8d2bfff700 1 -- 172.21.15.100:0/2766246170 wait complete. 2024-08-17T02:22:05.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 Processor -- start 2024-08-17T02:22:05.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 -- start start 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c067030 0x7f8d2c0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d2c0cfb00 0x7f8d2c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c158cc0 0x7f8d2c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0ca2b0 con 0x7f8d2c158cc0 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c96a0 con 0x7f8d2c067030 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c9980 con 0x7f8d2c0cfb00 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c158cc0 0x7f8d2c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c158cc0 0x7f8d2c15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56640/0 (socket says 172.21.15.100:56640) 2024-08-17T02:22:05.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 -- 172.21.15.100:0/1140754845 learned_addr learned my addr 172.21.15.100:0/1140754845 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2a7fc700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d2c0cfb00 0x7f8d2c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2affd700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c067030 0x7f8d2c0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 -- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d2c0cfb00 msgr2=0x7f8d2c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d2c0cfb00 0x7f8d2c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 -- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c067030 msgr2=0x7f8d2c0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c067030 0x7f8d2c0ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 -- 172.21.15.100:0/1140754845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140094f0 con 0x7f8d2c158cc0 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2a7fc700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d2c0cfb00 0x7f8d2c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2b7fe700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c158cc0 0x7f8d2c15a5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c4c0 tx=0x7f8d1c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d22ffd700 1 -- 172.21.15.100:0/1140754845 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c011590 con 0x7f8d2c158cc0 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d22ffd700 1 -- 172.21.15.100:0/1140754845 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c011710 con 0x7f8d2c158cc0 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d22ffd700 1 -- 172.21.15.100:0/1140754845 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c011ad0 con 0x7f8d2c158cc0 2024-08-17T02:22:05.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c0c9c60 con 0x7f8d2c158cc0 2024-08-17T02:22:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.865+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c0c92c0 con 0x7f8d2c158cc0 2024-08-17T02:22:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.869+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c0cac80 con 0x7f8d2c158cc0 2024-08-17T02:22:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.869+0000 7f8d22ffd700 1 -- 172.21.15.100:0/1140754845 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8d1c01b3b0 con 0x7f8d2c158cc0 2024-08-17T02:22:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.869+0000 7f8d22ffd700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d0c060860 0x7f8d0c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.869+0000 7f8d22ffd700 1 -- 172.21.15.100:0/1140754845 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d1c07bf90 con 0x7f8d2c158cc0 2024-08-17T02:22:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.869+0000 7f8d2affd700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d0c060860 0x7f8d0c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.869+0000 7f8d2affd700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d0c060860 0x7f8d0c062d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8d24007fd0 tx=0x7f8d240016f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:05.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:05.877+0000 7f8d22ffd700 1 -- 172.21.15.100:0/1140754845 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8d1c04c3e0 con 0x7f8d2c158cc0 2024-08-17T02:22:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.077+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8d2c0638f0 con 0x7f8d2c158cc0 2024-08-17T02:22:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.137+0000 7f8d22ffd700 1 -- 172.21.15.100:0/1140754845 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8d1c017020 con 0x7f8d2c158cc0 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d0c060860 msgr2=0x7f8d0c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d0c060860 0x7f8d0c062d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8d24007fd0 tx=0x7f8d240016f0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c158cc0 msgr2=0x7f8d2c15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c158cc0 0x7f8d2c15a5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c4c0 tx=0x7f8d1c007f50 comp rx=0 tx=0).stop 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 shutdown_connections 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d0c060860 0x7f8d0c062d10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c158cc0 0x7f8d2c15a5a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d2c0cfb00 0x7f8d2c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 --2- 172.21.15.100:0/1140754845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c067030 0x7f8d2c0ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 >> 172.21.15.100:0/1140754845 conn(0x7f8d2c0d0b50 msgr2=0x7f8d2c153f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 shutdown_connections 2024-08-17T02:22:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.141+0000 7f8d2bfff700 1 -- 172.21.15.100:0/1140754845 wait complete. 2024-08-17T02:22:06.158 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-17T02:22:06.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8f828700 1 Processor -- start 2024-08-17T02:22:06.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8f828700 1 -- start start 2024-08-17T02:22:06.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8f828700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a881554c0 0x7f7a880699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:06.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8f828700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8806b000 con 0x7f7a88068db0 2024-08-17T02:22:06.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8f828700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8806a3f0 con 0x7f7a88069ee0 2024-08-17T02:22:06.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8f828700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8806a6d0 con 0x7f7a881554c0 2024-08-17T02:22:06.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8ddc5700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7a88068db0 0x7f7a8806b490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46946/0 (socket says 172.21.15.100:46946) 2024-08-17T02:22:06.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8ddc5700 1 -- 172.21.15.100:0/205941619 learned_addr learned my addr 172.21.15.100:0/205941619 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:06.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.269+0000 7f7a8d5c4700 1 --2- 172.21.15.100:0/205941619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a881554c0 0x7f7a880699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:06.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a7ffff700 1 -- 172.21.15.100:0/205941619 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170745468 0 0) 0x7f7a8806a3f0 con 0x7f7a88069ee0 2024-08-17T02:22:06.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a7ffff700 1 -- 172.21.15.100:0/205941619 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a74003570 con 0x7f7a88069ee0 2024-08-17T02:22:06.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a7ffff700 1 -- 172.21.15.100:0/205941619 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242106947 0 0) 0x7f7a8806b000 con 0x7f7a88068db0 2024-08-17T02:22:06.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a7ffff700 1 -- 172.21.15.100:0/205941619 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a8806a3f0 con 0x7f7a88068db0 2024-08-17T02:22:06.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8d5c4700 1 -- 172.21.15.100:0/205941619 >> v1:172.21.15.123:6789/0 conn(0x7f7a88069ee0 legacy=0x7f7a880686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:06.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8d5c4700 1 -- 172.21.15.100:0/205941619 >> v1:172.21.15.100:6789/0 conn(0x7f7a88068db0 legacy=0x7f7a8806b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:06.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8d5c4700 1 -- 172.21.15.100:0/205941619 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8806a9b0 con 0x7f7a881554c0 2024-08-17T02:22:06.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8d5c4700 1 --2- 172.21.15.100:0/205941619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a881554c0 0x7f7a880699a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009910 tx=0x7f7a840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a41736cecfc438b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:06.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a7ffff700 1 -- 172.21.15.100:0/205941619 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84011630 con 0x7f7a881554c0 2024-08-17T02:22:06.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a7ffff700 1 -- 172.21.15.100:0/205941619 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a84011790 con 0x7f7a881554c0 2024-08-17T02:22:06.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a7ffff700 1 -- 172.21.15.100:0/205941619 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84011b10 con 0x7f7a881554c0 2024-08-17T02:22:06.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8f828700 1 -- 172.21.15.100:0/205941619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a881554c0 msgr2=0x7f7a880699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:06.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8f828700 1 --2- 172.21.15.100:0/205941619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a881554c0 0x7f7a880699a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009910 tx=0x7f7a840091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8f828700 1 -- 172.21.15.100:0/205941619 shutdown_connections 2024-08-17T02:22:06.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8f828700 1 --2- 172.21.15.100:0/205941619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a881554c0 0x7f7a880699a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8f828700 1 -- 172.21.15.100:0/205941619 >> 172.21.15.100:0/205941619 conn(0x7f7a880c9bf0 msgr2=0x7f7a8806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:06.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8f828700 1 -- 172.21.15.100:0/205941619 shutdown_connections 2024-08-17T02:22:06.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.273+0000 7f7a8f828700 1 -- 172.21.15.100:0/205941619 wait complete. 2024-08-17T02:22:06.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 Processor -- start 2024-08-17T02:22:06.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 -- start start 2024-08-17T02:22:06.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88068db0 0x7f7a880aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:06.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a88069ee0 0x7f7a880a8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a881554c0 0x7f7a88156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880aad10 con 0x7f7a88068db0 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880a8980 con 0x7f7a881554c0 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880a8c60 con 0x7f7a88069ee0 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88068db0 0x7f7a880aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88068db0 0x7f7a880aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56648/0 (socket says 172.21.15.100:56648) 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 -- 172.21.15.100:0/204326902 learned_addr learned my addr 172.21.15.100:0/204326902 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8ddc5700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a881554c0 0x7f7a88156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8cdc3700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a88069ee0 0x7f7a880a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 -- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a88069ee0 msgr2=0x7f7a880a8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a88069ee0 0x7f7a880a8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 -- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a881554c0 msgr2=0x7f7a88156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:06.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a881554c0 0x7f7a88156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:06.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 -- 172.21.15.100:0/204326902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84009420 con 0x7f7a88068db0 2024-08-17T02:22:06.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8cdc3700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a88069ee0 0x7f7a880a8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:06.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8d5c4700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88068db0 0x7f7a880aa7d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009860 tx=0x7f7a84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:06.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a7d7fa700 1 -- 172.21.15.100:0/204326902 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84000d10 con 0x7f7a88068db0 2024-08-17T02:22:06.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a7d7fa700 1 -- 172.21.15.100:0/204326902 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a84011900 con 0x7f7a88068db0 2024-08-17T02:22:06.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a880a8f40 con 0x7f7a88068db0 2024-08-17T02:22:06.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a7d7fa700 1 -- 172.21.15.100:0/204326902 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84025650 con 0x7f7a88068db0 2024-08-17T02:22:06.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.277+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a880a54f0 con 0x7f7a88068db0 2024-08-17T02:22:06.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.281+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a880645c0 con 0x7f7a88068db0 2024-08-17T02:22:06.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.281+0000 7f7a7d7fa700 1 -- 172.21.15.100:0/204326902 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7a84023070 con 0x7f7a88068db0 2024-08-17T02:22:06.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.281+0000 7f7a7d7fa700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7a6c060860 0x7f7a6c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:06.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.281+0000 7f7a7d7fa700 1 -- 172.21.15.100:0/204326902 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7a8407dec0 con 0x7f7a88068db0 2024-08-17T02:22:06.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.281+0000 7f7a8cdc3700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7a6c060860 0x7f7a6c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:06.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.281+0000 7f7a8cdc3700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7a6c060860 0x7f7a6c062d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a88159d50 tx=0x7f7a78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:06.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.289+0000 7f7a7d7fa700 1 -- 172.21.15.100:0/204326902 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7a8404e310 con 0x7f7a88068db0 2024-08-17T02:22:06.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:06.493+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a8815a200 con 0x7f7a88068db0 2024-08-17T02:22:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.144+0000 7f7a7d7fa700 1 -- 172.21.15.100:0/204326902 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a840150c0 con 0x7f7a88068db0 2024-08-17T02:22:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7a6c060860 msgr2=0x7f7a6c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7a6c060860 0x7f7a6c062d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a88159d50 tx=0x7f7a78009040 comp rx=0 tx=0).stop 2024-08-17T02:22:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88068db0 msgr2=0x7f7a880aa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88068db0 0x7f7a880aa7d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009860 tx=0x7f7a84017000 comp rx=0 tx=0).stop 2024-08-17T02:22:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 shutdown_connections 2024-08-17T02:22:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7a6c060860 0x7f7a6c062d10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a881554c0 0x7f7a88156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a88069ee0 0x7f7a880a8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 --2- 172.21.15.100:0/204326902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88068db0 0x7f7a880aa7d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 >> 172.21.15.100:0/204326902 conn(0x7f7a880c9bf0 msgr2=0x7f7a880afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:07.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 shutdown_connections 2024-08-17T02:22:07.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.148+0000 7f7a8f828700 1 -- 172.21.15.100:0/204326902 wait complete. 2024-08-17T02:22:07.168 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.276+0000 7fa066ea9700 1 Processor -- start 2024-08-17T02:22:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.276+0000 7fa066ea9700 1 -- start start 2024-08-17T02:22:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.276+0000 7fa066ea9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa060156d40 0x7fa060157110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.276+0000 7fa066ea9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa06006d2f0 0x7fa06006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.276+0000 7fa066ea9700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0600addb0 con 0x7fa06015af40 2024-08-17T02:22:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.276+0000 7fa066ea9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06015bab0 con 0x7fa06006d2f0 2024-08-17T02:22:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.276+0000 7fa066ea9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600ae930 con 0x7fa060156d40 2024-08-17T02:22:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa065446700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa06015af40 0x7fa06015b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46948/0 (socket says 172.21.15.100:46948) 2024-08-17T02:22:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa065446700 1 -- 172.21.15.100:0/1284370857 learned_addr learned my addr 172.21.15.100:0/1284370857 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa064c45700 1 --2- 172.21.15.100:0/1284370857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa060156d40 0x7fa060157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05ffff700 1 --2- 172.21.15.100:0/1284370857 >> v2:172.21.15.123:3300/0 conn(0x7fa06006d2f0 0x7fa06006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05f7fe700 1 -- 172.21.15.100:0/1284370857 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326783961 0 0) 0x7fa0600addb0 con 0x7fa06015af40 2024-08-17T02:22:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05f7fe700 1 -- 172.21.15.100:0/1284370857 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa040003570 con 0x7fa06015af40 2024-08-17T02:22:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05ffff700 1 -- 172.21.15.100:0/1284370857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa060156d40 msgr2=0x7fa060157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05ffff700 1 --2- 172.21.15.100:0/1284370857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa060156d40 0x7fa060157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05ffff700 1 -- 172.21.15.100:0/1284370857 >> v1:172.21.15.100:6789/0 conn(0x7fa06015af40 legacy=0x7fa06015b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05ffff700 1 -- 172.21.15.100:0/1284370857 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06006dc80 con 0x7fa06006d2f0 2024-08-17T02:22:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05ffff700 1 --2- 172.21.15.100:0/1284370857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06006d2f0 0x7fa06006d740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098e0 tx=0x7fa0480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=355d740b3f6627dd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05f7fe700 1 -- 172.21.15.100:0/1284370857 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048011630 con 0x7fa06006d2f0 2024-08-17T02:22:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05f7fe700 1 -- 172.21.15.100:0/1284370857 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa048011790 con 0x7fa06006d2f0 2024-08-17T02:22:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa05f7fe700 1 -- 172.21.15.100:0/1284370857 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048011b30 con 0x7fa06006d2f0 2024-08-17T02:22:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa066ea9700 1 -- 172.21.15.100:0/1284370857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06006d2f0 msgr2=0x7fa06006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa066ea9700 1 --2- 172.21.15.100:0/1284370857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06006d2f0 0x7fa06006d740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098e0 tx=0x7fa0480091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa066ea9700 1 -- 172.21.15.100:0/1284370857 shutdown_connections 2024-08-17T02:22:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa066ea9700 1 --2- 172.21.15.100:0/1284370857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06006d2f0 0x7fa06006d740 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa066ea9700 1 --2- 172.21.15.100:0/1284370857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa060156d40 0x7fa060157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa066ea9700 1 -- 172.21.15.100:0/1284370857 >> 172.21.15.100:0/1284370857 conn(0x7fa0600ceb30 msgr2=0x7fa06006cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa066ea9700 1 -- 172.21.15.100:0/1284370857 shutdown_connections 2024-08-17T02:22:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.280+0000 7fa066ea9700 1 -- 172.21.15.100:0/1284370857 wait complete. 2024-08-17T02:22:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.284+0000 7fa066ea9700 1 Processor -- start 2024-08-17T02:22:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.284+0000 7fa066ea9700 1 -- start start 2024-08-17T02:22:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06006d2f0 0x7fa0600691f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa060156d40 0x7fa0600c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa06015af40 0x7fa060169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060069730 con 0x7fa06006d2f0 2024-08-17T02:22:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600687a0 con 0x7fa060156d40 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa060068d60 con 0x7fa06015af40 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06006d2f0 0x7fa0600691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06006d2f0 0x7fa0600691f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56660/0 (socket says 172.21.15.100:56660) 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 -- 172.21.15.100:0/3099612056 learned_addr learned my addr 172.21.15.100:0/3099612056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa05ffff700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa060156d40 0x7fa0600c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa065446700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa06015af40 0x7fa060169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 -- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa06015af40 msgr2=0x7fa060169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa06015af40 0x7fa060169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 -- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa060156d40 msgr2=0x7fa0600c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa060156d40 0x7fa0600c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 -- 172.21.15.100:0/3099612056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048009420 con 0x7fa06006d2f0 2024-08-17T02:22:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa064c45700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06006d2f0 0x7fa0600691f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa05000ecb0 tx=0x7fa0500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa05cff9700 1 -- 172.21.15.100:0/3099612056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000c850 con 0x7fa06006d2f0 2024-08-17T02:22:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa05cff9700 1 -- 172.21.15.100:0/3099612056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05000c9b0 con 0x7fa06006d2f0 2024-08-17T02:22:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa05cff9700 1 -- 172.21.15.100:0/3099612056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000cd50 con 0x7fa06006d2f0 2024-08-17T02:22:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06016a490 con 0x7fa06006d2f0 2024-08-17T02:22:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0600689f0 con 0x7fa06006d2f0 2024-08-17T02:22:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa05cff9700 1 -- 172.21.15.100:0/3099612056 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa050013680 con 0x7fa06006d2f0 2024-08-17T02:22:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.288+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0600645c0 con 0x7fa06006d2f0 2024-08-17T02:22:07.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.292+0000 7fa05cff9700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa044060860 0x7fa044062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.292+0000 7fa05cff9700 1 -- 172.21.15.100:0/3099612056 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa05007cd60 con 0x7fa06006d2f0 2024-08-17T02:22:07.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.292+0000 7fa05ffff700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa044060860 0x7fa044062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.292+0000 7fa05ffff700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa044060860 0x7fa044062d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0480091e0 tx=0x7fa048017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.296+0000 7fa05cff9700 1 -- 172.21.15.100:0/3099612056 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa05004d1b0 con 0x7fa06006d2f0 2024-08-17T02:22:07.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.496+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0600cb1a0 con 0x7fa06006d2f0 2024-08-17T02:22:07.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.496+0000 7fa05cff9700 1 -- 172.21.15.100:0/3099612056 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fa0500507d0 con 0x7fa06006d2f0 2024-08-17T02:22:07.501 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:07.501 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:04.459540+0000","modified":"2024-08-17T02:22:07.136998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa044060860 msgr2=0x7fa044062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa044060860 0x7fa044062d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0480091e0 tx=0x7fa048017000 comp rx=0 tx=0).stop 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06006d2f0 msgr2=0x7fa0600691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06006d2f0 0x7fa0600691f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa05000ecb0 tx=0x7fa0500091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 shutdown_connections 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa044060860 0x7fa044062d10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa06015af40 0x7fa060169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa060156d40 0x7fa0600c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 --2- 172.21.15.100:0/3099612056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06006d2f0 0x7fa0600691f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 >> 172.21.15.100:0/3099612056 conn(0x7fa0600ceb30 msgr2=0x7fa060150550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 shutdown_connections 2024-08-17T02:22:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.500+0000 7fa066ea9700 1 -- 172.21.15.100:0/3099612056 wait complete. 2024-08-17T02:22:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-08-17T02:22:07.515 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:07.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.616+0000 7fb174953700 1 Processor -- start 2024-08-17T02:22:07.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.616+0000 7fb174953700 1 -- start start 2024-08-17T02:22:07.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb17015c620 0x7fb1700b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1700ad1b0 0x7fb1700c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1700c86e0 con 0x7fb17015cd90 2024-08-17T02:22:07.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17015d210 con 0x7fb17015c620 2024-08-17T02:22:07.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700cb850 con 0x7fb1700ad1b0 2024-08-17T02:22:07.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1700ad1b0 0x7fb1700c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1700ad1b0 0x7fb1700c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:51510/0 (socket says 172.21.15.100:51510) 2024-08-17T02:22:07.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16f7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb17015c620 0x7fb1700b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16ffff700 1 -- 172.21.15.100:0/2708567988 learned_addr learned my addr 172.21.15.100:0/2708567988 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:07.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16ffff700 1 -- 172.21.15.100:0/2708567988 >> v2:172.21.15.123:3300/0 conn(0x7fb17015c620 msgr2=0x7fb1700b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:07.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16ffff700 1 --2- 172.21.15.100:0/2708567988 >> v2:172.21.15.123:3300/0 conn(0x7fb17015c620 0x7fb1700b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16ffff700 1 -- 172.21.15.100:0/2708567988 >> v1:172.21.15.123:6789/0 conn(0x7fb17015cd90 legacy=0x7fb1700acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16ffff700 1 -- 172.21.15.100:0/2708567988 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17015d490 con 0x7fb1700ad1b0 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16ffff700 1 --2- 172.21.15.100:0/2708567988 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1700ad1b0 0x7fb1700c8b70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb1680099b0 tx=0x7fb168009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3064586ed6431206 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16e7fc700 1 -- 172.21.15.100:0/2708567988 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168009ea0 con 0x7fb1700ad1b0 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16e7fc700 1 -- 172.21.15.100:0/2708567988 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16800b860 con 0x7fb1700ad1b0 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb16e7fc700 1 -- 172.21.15.100:0/2708567988 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16800bc00 con 0x7fb1700ad1b0 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 -- 172.21.15.100:0/2708567988 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1700ad1b0 msgr2=0x7fb1700c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 --2- 172.21.15.100:0/2708567988 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1700ad1b0 0x7fb1700c8b70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb1680099b0 tx=0x7fb168009250 comp rx=0 tx=0).stop 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 -- 172.21.15.100:0/2708567988 shutdown_connections 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 --2- 172.21.15.100:0/2708567988 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1700ad1b0 0x7fb1700c8b70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 --2- 172.21.15.100:0/2708567988 >> v2:172.21.15.123:3300/0 conn(0x7fb17015c620 0x7fb1700b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 -- 172.21.15.100:0/2708567988 >> 172.21.15.100:0/2708567988 conn(0x7fb1700ca900 msgr2=0x7fb1700cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 -- 172.21.15.100:0/2708567988 shutdown_connections 2024-08-17T02:22:07.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 -- 172.21.15.100:0/2708567988 wait complete. 2024-08-17T02:22:07.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.620+0000 7fb174953700 1 Processor -- start 2024-08-17T02:22:07.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 -- start start 2024-08-17T02:22:07.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1700ad1b0 0x7fb17016c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb17015c620 0x7fb1700c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb17015cd90 0x7fb1700c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700c6730 con 0x7fb1700ad1b0 2024-08-17T02:22:07.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700aabc0 con 0x7fb17015c620 2024-08-17T02:22:07.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17016c670 con 0x7fb17015cd90 2024-08-17T02:22:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1700ad1b0 0x7fb17016c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1700ad1b0 0x7fb17016c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56670/0 (socket says 172.21.15.100:56670) 2024-08-17T02:22:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 -- 172.21.15.100:0/4183801414 learned_addr learned my addr 172.21.15.100:0/4183801414 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16ffff700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb17015cd90 0x7fb1700c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16effd700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb17015c620 0x7fb1700c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 -- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb17015cd90 msgr2=0x7fb1700c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb17015cd90 0x7fb1700c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 -- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb17015c620 msgr2=0x7fb1700c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb17015c620 0x7fb1700c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 -- 172.21.15.100:0/4183801414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680094f0 con 0x7fb1700ad1b0 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16effd700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb17015c620 0x7fb1700c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb16f7fe700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1700ad1b0 0x7fb17016c130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb16000ec80 tx=0x7fb1600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb1677fe700 1 -- 172.21.15.100:0/4183801414 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000c850 con 0x7fb1700ad1b0 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb1677fe700 1 -- 172.21.15.100:0/4183801414 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16000c9b0 con 0x7fb1700ad1b0 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb1677fe700 1 -- 172.21.15.100:0/4183801414 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000cd50 con 0x7fb1700ad1b0 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1700c5b20 con 0x7fb1700ad1b0 2024-08-17T02:22:07.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.624+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1700c60e0 con 0x7fb1700ad1b0 2024-08-17T02:22:07.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.628+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1700643a0 con 0x7fb1700ad1b0 2024-08-17T02:22:07.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.628+0000 7fb1677fe700 1 -- 172.21.15.100:0/4183801414 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb160013680 con 0x7fb1700ad1b0 2024-08-17T02:22:07.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.628+0000 7fb1677fe700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb150060860 0x7fb150062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.628+0000 7fb1677fe700 1 -- 172.21.15.100:0/4183801414 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb16007ce30 con 0x7fb1700ad1b0 2024-08-17T02:22:07.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.628+0000 7fb16effd700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb150060860 0x7fb150062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.628+0000 7fb16effd700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb150060860 0x7fb150062d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb1700c71f0 tx=0x7fb158009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.636+0000 7fb1677fe700 1 -- 172.21.15.100:0/4183801414 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb16004d280 con 0x7fb1700ad1b0 2024-08-17T02:22:07.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.804+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1700681a0 con 0x7fb1700ad1b0 2024-08-17T02:22:07.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.804+0000 7fb1677fe700 1 -- 172.21.15.100:0/4183801414 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fb1600508a0 con 0x7fb1700ad1b0 2024-08-17T02:22:07.809 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:07.809 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:07.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb150060860 msgr2=0x7fb150062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb150060860 0x7fb150062d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb1700c71f0 tx=0x7fb158009040 comp rx=0 tx=0).stop 2024-08-17T02:22:07.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1700ad1b0 msgr2=0x7fb17016c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1700ad1b0 0x7fb17016c130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb16000ec80 tx=0x7fb1600091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 shutdown_connections 2024-08-17T02:22:07.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb150060860 0x7fb150062d10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb17015cd90 0x7fb1700c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb17015c620 0x7fb1700c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 --2- 172.21.15.100:0/4183801414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1700ad1b0 0x7fb17016c130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 >> 172.21.15.100:0/4183801414 conn(0x7fb1700ca900 msgr2=0x7fb170159990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:07.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 shutdown_connections 2024-08-17T02:22:07.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.808+0000 7fb174953700 1 -- 172.21.15.100:0/4183801414 wait complete. 2024-08-17T02:22:07.823 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-17T02:22:07.823 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:07.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 Processor -- start 2024-08-17T02:22:07.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- start start 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f571015a0f0 0x7f57100c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f571015a4c0 0x7f57100c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571015d830 con 0x7f57100680b0 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571015cf00 con 0x7f571015a4c0 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571015d1e0 con 0x7f571015a0f0 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5714e21700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f571015a0f0 0x7f57100c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5714e21700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f571015a0f0 0x7f57100c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:51532/0 (socket says 172.21.15.100:51532) 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5714e21700 1 -- 172.21.15.100:0/3514667715 learned_addr learned my addr 172.21.15.100:0/3514667715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:07.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5707fff700 1 --2- 172.21.15.100:0/3514667715 >> v2:172.21.15.123:3300/0 conn(0x7f571015a4c0 0x7f57100c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5714e21700 1 -- 172.21.15.100:0/3514667715 >> v2:172.21.15.123:3300/0 conn(0x7f571015a4c0 msgr2=0x7f57100c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:07.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5714e21700 1 --2- 172.21.15.100:0/3514667715 >> v2:172.21.15.123:3300/0 conn(0x7f571015a4c0 0x7f57100c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5714e21700 1 -- 172.21.15.100:0/3514667715 >> v1:172.21.15.123:6789/0 conn(0x7f57100680b0 legacy=0x7f5710066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f57077fe700 1 -- 172.21.15.100:0/3514667715 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122657192 0 0) 0x7f571015d830 con 0x7f57100680b0 2024-08-17T02:22:07.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5714e21700 1 -- 172.21.15.100:0/3514667715 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571015d4c0 con 0x7f571015a0f0 2024-08-17T02:22:07.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5714e21700 1 --2- 172.21.15.100:0/3514667715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f571015a0f0 0x7f57100c96c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f570c0099b0 tx=0x7f570c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abbb4f567dda531 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f57077fe700 1 -- 172.21.15.100:0/3514667715 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c009ea0 con 0x7f571015a0f0 2024-08-17T02:22:07.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f57077fe700 1 -- 172.21.15.100:0/3514667715 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c00b860 con 0x7f571015a0f0 2024-08-17T02:22:07.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f57077fe700 1 -- 172.21.15.100:0/3514667715 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c00bc00 con 0x7f571015a0f0 2024-08-17T02:22:07.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- 172.21.15.100:0/3514667715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f571015a0f0 msgr2=0x7f57100c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 --2- 172.21.15.100:0/3514667715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f571015a0f0 0x7f57100c96c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f570c0099b0 tx=0x7f570c009250 comp rx=0 tx=0).stop 2024-08-17T02:22:07.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- 172.21.15.100:0/3514667715 shutdown_connections 2024-08-17T02:22:07.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 --2- 172.21.15.100:0/3514667715 >> v2:172.21.15.123:3300/0 conn(0x7f571015a4c0 0x7f57100c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 --2- 172.21.15.100:0/3514667715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f571015a0f0 0x7f57100c96c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- 172.21.15.100:0/3514667715 >> 172.21.15.100:0/3514667715 conn(0x7f57100ccfc0 msgr2=0x7f57100c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:07.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- 172.21.15.100:0/3514667715 shutdown_connections 2024-08-17T02:22:07.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.932+0000 7f5715e23700 1 -- 172.21.15.100:0/3514667715 wait complete. 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 Processor -- start 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 -- start start 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57100680b0 0x7f571014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571015a0f0 0x7f5710156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f571015a4c0 0x7f5710152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5710153e60 con 0x7f571015a0f0 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571015c960 con 0x7f571015a4c0 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57101540b0 con 0x7f57100680b0 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571015a0f0 0x7f5710156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5714e21700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57100680b0 0x7f571014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571015a0f0 0x7f5710156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56674/0 (socket says 172.21.15.100:56674) 2024-08-17T02:22:07.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 -- 172.21.15.100:0/2386651183 learned_addr learned my addr 172.21.15.100:0/2386651183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:07.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715622700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f571015a4c0 0x7f5710152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 -- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57100680b0 msgr2=0x7f571014b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57100680b0 0x7f571014b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 -- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f571015a4c0 msgr2=0x7f5710152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:07.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f571015a4c0 0x7f5710152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:07.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 -- 172.21.15.100:0/2386651183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c0094f0 con 0x7f571015a0f0 2024-08-17T02:22:07.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5707fff700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571015a0f0 0x7f5710156790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f570000e950 tx=0x7f57000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5704ff9700 1 -- 172.21.15.100:0/2386651183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700013680 con 0x7f571015a0f0 2024-08-17T02:22:07.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5704ff9700 1 -- 172.21.15.100:0/2386651183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57000137e0 con 0x7f571015a0f0 2024-08-17T02:22:07.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5710154390 con 0x7f571015a0f0 2024-08-17T02:22:07.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5704ff9700 1 -- 172.21.15.100:0/2386651183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700013ba0 con 0x7f571015a0f0 2024-08-17T02:22:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.936+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571014edb0 con 0x7f571015a0f0 2024-08-17T02:22:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.940+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57100638f0 con 0x7f571015a0f0 2024-08-17T02:22:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.940+0000 7f5704ff9700 1 -- 172.21.15.100:0/2386651183 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f57000123d0 con 0x7f571015a0f0 2024-08-17T02:22:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.940+0000 7f5704ff9700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f56fc060890 0x7f56fc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.940+0000 7f5704ff9700 1 -- 172.21.15.100:0/2386651183 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f570007cd40 con 0x7f571015a0f0 2024-08-17T02:22:07.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.944+0000 7f5714e21700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f56fc060890 0x7f56fc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:07.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.944+0000 7f5714e21700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f56fc060890 0x7f56fc062d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f570c0099b0 tx=0x7f570c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:07.948+0000 7f5704ff9700 1 -- 172.21.15.100:0/2386651183 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f570004d190 con 0x7f571015a0f0 2024-08-17T02:22:08.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.108+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f571006ea50 con 0x7f56fc060890 2024-08-17T02:22:08.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.124+0000 7f5704ff9700 1 -- 172.21.15.100:0/2386651183 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252213 (secure 0 0 0) 0x7f571006ea50 con 0x7f56fc060890 2024-08-17T02:22:08.131 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f56fc060890 msgr2=0x7f56fc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f56fc060890 0x7f56fc062d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f570c0099b0 tx=0x7f570c017000 comp rx=0 tx=0).stop 2024-08-17T02:22:08.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571015a0f0 msgr2=0x7f5710156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571015a0f0 0x7f5710156790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f570000e950 tx=0x7f57000091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 shutdown_connections 2024-08-17T02:22:08.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f56fc060890 0x7f56fc062d40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f571015a4c0 0x7f5710152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571015a0f0 0x7f5710156790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 --2- 172.21.15.100:0/2386651183 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57100680b0 0x7f571014b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 >> 172.21.15.100:0/2386651183 conn(0x7f57100ccfc0 msgr2=0x7f5710151b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:08.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 shutdown_connections 2024-08-17T02:22:08.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.128+0000 7f5715e23700 1 -- 172.21.15.100:0/2386651183 wait complete. 2024-08-17T02:22:08.137 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:08.146 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-08-17T02:22:07.221728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381784,"kb_used_data":2536,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592936,"statfs":{"total":773094113280,"available":770655166464,"internally_reserved":0,"allocated":2596864,"data_stored":1607946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.120334"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.099961+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100907+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100678+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095631+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:28:53.644529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101508+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:07:22.383055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096031+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103503+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:07:22.383055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100784+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100245+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:50.168021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.097451+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100709+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095296+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.094655+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095796+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101956+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096039+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":160,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-17T02:22:02.815000+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:02.815000+0000","last_peered":"2024-08-17T02:22:02.815000+0000","last_clean":"2024-08-17T02:22:02.815000+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:02.815000+0000","last_undegraded":"2024-08-17T02:22:02.815000+0000","last_fullsized":"2024-08-17T02:22:02.815000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095996+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:11.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.104598+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096212+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096355+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100878+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103730+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101835+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100653+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.105400+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:48:47.160779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100688+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101002+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103972+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103678+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.104331+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101123+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101303+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:28:53.644529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298308,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073532,"statfs":{"total":96636764160,"available":96331296768,"internally_reserved":0,"allocated":782336,"data_stored":649428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.831}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]}]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298308,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073532,"statfs":{"total":96636764160,"available":96331296768,"internally_reserved":0,"allocated":782336,"data_stored":649428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:08.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:08.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4ee752700 1 Processor -- start 2024-08-17T02:22:08.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4ee752700 1 -- start start 2024-08-17T02:22:08.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4ee752700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb4e815d540 0x7fb4e80af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4ee752700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e80b0270 con 0x7fb4e80a9430 2024-08-17T02:22:08.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4ee752700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e80af660 con 0x7fb4e806e830 2024-08-17T02:22:08.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4ee752700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e80af940 con 0x7fb4e815d540 2024-08-17T02:22:08.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e7fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb4e815d540 0x7fb4e80af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4eccef700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb4e80a9430 0x7fb4e80b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46952/0 (socket says 172.21.15.100:46952) 2024-08-17T02:22:08.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4eccef700 1 -- 172.21.15.100:0/2338948368 learned_addr learned my addr 172.21.15.100:0/2338948368 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:08.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e6ffd700 1 -- 172.21.15.100:0/2338948368 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156525830 0 0) 0x7fb4e80b0270 con 0x7fb4e80a9430 2024-08-17T02:22:08.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e6ffd700 1 -- 172.21.15.100:0/2338948368 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4c8003570 con 0x7fb4e80a9430 2024-08-17T02:22:08.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e7fff700 1 -- 172.21.15.100:0/2338948368 >> v1:172.21.15.123:6789/0 conn(0x7fb4e806e830 legacy=0x7fb4e80a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e7fff700 1 -- 172.21.15.100:0/2338948368 >> v1:172.21.15.100:6789/0 conn(0x7fb4e80a9430 legacy=0x7fb4e80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e6ffd700 1 -- 172.21.15.100:0/2338948368 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1575208323 0 0) 0x7fb4c8003570 con 0x7fb4e80a9430 2024-08-17T02:22:08.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e7fff700 1 -- 172.21.15.100:0/2338948368 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e80afc20 con 0x7fb4e815d540 2024-08-17T02:22:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e7fff700 1 --2- 172.21.15.100:0/2338948368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e815d540 0x7fb4e80af030 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00098e0 tx=0x7fb4d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1206a1609b5120 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e6ffd700 1 -- 172.21.15.100:0/2338948368 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00017f0 con 0x7fb4e815d540 2024-08-17T02:22:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e6ffd700 1 -- 172.21.15.100:0/2338948368 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d0011420 con 0x7fb4e815d540 2024-08-17T02:22:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.256+0000 7fb4e6ffd700 1 -- 172.21.15.100:0/2338948368 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00117a0 con 0x7fb4e815d540 2024-08-17T02:22:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- 172.21.15.100:0/2338948368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e815d540 msgr2=0x7fb4e80af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 --2- 172.21.15.100:0/2338948368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e815d540 0x7fb4e80af030 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00098e0 tx=0x7fb4d00091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- 172.21.15.100:0/2338948368 shutdown_connections 2024-08-17T02:22:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 --2- 172.21.15.100:0/2338948368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e815d540 0x7fb4e80af030 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- 172.21.15.100:0/2338948368 >> 172.21.15.100:0/2338948368 conn(0x7fb4e8066e00 msgr2=0x7fb4e8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- 172.21.15.100:0/2338948368 shutdown_connections 2024-08-17T02:22:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- 172.21.15.100:0/2338948368 wait complete. 2024-08-17T02:22:08.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 Processor -- start 2024-08-17T02:22:08.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- start start 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e806e830 0x7fb4e814cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e80a9430 0x7fb4e816dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e815d540 0x7fb4e80ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e816ec30 con 0x7fb4e806e830 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e816e670 con 0x7fb4e80a9430 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.260+0000 7fb4ee752700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e816e950 con 0x7fb4e815d540 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e806e830 0x7fb4e814cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e806e830 0x7fb4e814cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56694/0 (socket says 172.21.15.100:56694) 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 -- 172.21.15.100:0/3578900071 learned_addr learned my addr 172.21.15.100:0/3578900071 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4eccef700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e815d540 0x7fb4e80ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e77fe700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e80a9430 0x7fb4e816dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 -- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e815d540 msgr2=0x7fb4e80ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e815d540 0x7fb4e80ab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 -- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e80a9430 msgr2=0x7fb4e816dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e80a9430 0x7fb4e816dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 -- 172.21.15.100:0/3578900071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0009420 con 0x7fb4e806e830 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4e7fff700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e806e830 0x7fb4e814cfe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4e806f870 tx=0x7fb4d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4dffff700 1 -- 172.21.15.100:0/3578900071 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d001cb80 con 0x7fb4e806e830 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4dffff700 1 -- 172.21.15.100:0/3578900071 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d001cce0 con 0x7fb4e806e830 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e80aad40 con 0x7fb4e806e830 2024-08-17T02:22:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4dffff700 1 -- 172.21.15.100:0/3578900071 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d001a6a0 con 0x7fb4e806e830 2024-08-17T02:22:08.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e814d520 con 0x7fb4e806e830 2024-08-17T02:22:08.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e80645c0 con 0x7fb4e806e830 2024-08-17T02:22:08.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.264+0000 7fb4dffff700 1 -- 172.21.15.100:0/3578900071 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb4d001aa90 con 0x7fb4e806e830 2024-08-17T02:22:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.268+0000 7fb4dffff700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4cc060860 0x7fb4cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.268+0000 7fb4dffff700 1 -- 172.21.15.100:0/3578900071 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb4d007e500 con 0x7fb4e806e830 2024-08-17T02:22:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.268+0000 7fb4e77fe700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4cc060860 0x7fb4cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.268+0000 7fb4e77fe700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4cc060860 0x7fb4cc062d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8099cc0 tx=0x7fb4d8007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.272+0000 7fb4dffff700 1 -- 172.21.15.100:0/3578900071 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb4d004e990 con 0x7fb4e806e830 2024-08-17T02:22:08.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.436+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4e80b11c0 con 0x7fb4cc060860 2024-08-17T02:22:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.448+0000 7fb4dffff700 1 -- 172.21.15.100:0/3578900071 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252213 (secure 0 0 0) 0x7fb4e80b11c0 con 0x7fb4cc060860 2024-08-17T02:22:08.454 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:08.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4cc060860 msgr2=0x7fb4cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4cc060860 0x7fb4cc062d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8099cc0 tx=0x7fb4d8007ec0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e806e830 msgr2=0x7fb4e814cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e806e830 0x7fb4e814cfe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4e806f870 tx=0x7fb4d0016000 comp rx=0 tx=0).stop 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 shutdown_connections 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4cc060860 0x7fb4cc062d10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e815d540 0x7fb4e80ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e80a9430 0x7fb4e816dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 --2- 172.21.15.100:0/3578900071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e806e830 0x7fb4e814cfe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 >> 172.21.15.100:0/3578900071 conn(0x7fb4e8066e00 msgr2=0x7fb4e806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 shutdown_connections 2024-08-17T02:22:08.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.452+0000 7fb4ee752700 1 -- 172.21.15.100:0/3578900071 wait complete. 2024-08-17T02:22:08.458 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:08.470 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-08-17T02:22:07.221728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381784,"kb_used_data":2536,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592936,"statfs":{"total":773094113280,"available":770655166464,"internally_reserved":0,"allocated":2596864,"data_stored":1607946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.120334"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.099961+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100907+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100678+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095631+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:28:53.644529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101508+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:07:22.383055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096031+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103503+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:07:22.383055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100784+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100245+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:50.168021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.097451+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100709+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095296+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.094655+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095796+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101956+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096039+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":160,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-17T02:22:02.815000+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:02.815000+0000","last_peered":"2024-08-17T02:22:02.815000+0000","last_clean":"2024-08-17T02:22:02.815000+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:02.815000+0000","last_undegraded":"2024-08-17T02:22:02.815000+0000","last_fullsized":"2024-08-17T02:22:02.815000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095996+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:11.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.104598+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096212+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096355+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100878+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103730+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101835+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100653+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.105400+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:48:47.160779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100688+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101002+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103972+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103678+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.104331+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101123+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101303+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:28:53.644529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298308,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073532,"statfs":{"total":96636764160,"available":96331296768,"internally_reserved":0,"allocated":782336,"data_stored":649428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.831}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]}]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298308,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073532,"statfs":{"total":96636764160,"available":96331296768,"internally_reserved":0,"allocated":782336,"data_stored":649428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:08.474 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:08.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6c05700 1 Processor -- start 2024-08-17T02:22:08.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6c05700 1 -- start start 2024-08-17T02:22:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6c05700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f30e006e720 0x7f30e006eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6c05700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e00ae8c0 con 0x7f30e013f670 2024-08-17T02:22:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6c05700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e00b1820 con 0x7f30e013fa40 2024-08-17T02:22:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6c05700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e006f0e0 con 0x7f30e006e720 2024-08-17T02:22:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f30e013f670 0x7f30e00b1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46956/0 (socket says 172.21.15.100:46956) 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 -- 172.21.15.100:0/261862552 learned_addr learned my addr 172.21.15.100:0/261862552 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6404700 1 --2- 172.21.15.100:0/261862552 >> v2:172.21.15.123:3300/0 conn(0x7f30e006e720 0x7f30e006eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246605499 0 0) 0x7f30e00ae8c0 con 0x7f30e013f670 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30c8003570 con 0x7f30e013f670 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986042062 0 0) 0x7f30e00b1820 con 0x7f30e013fa40 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e00ae8c0 con 0x7f30e013fa40 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 889920020 0 0) 0x7f30c8003570 con 0x7f30e013f670 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 >> v2:172.21.15.123:3300/0 conn(0x7f30e006e720 msgr2=0x7f30e006eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 --2- 172.21.15.100:0/261862552 >> v2:172.21.15.123:3300/0 conn(0x7f30e006e720 0x7f30e006eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 >> v1:172.21.15.123:6789/0 conn(0x7f30e013fa40 legacy=0x7f30e00ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00b23c0 con 0x7f30e013f670 2024-08-17T02:22:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f30d4001470 con 0x7f30e013f670 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 >> v1:172.21.15.100:6789/0 conn(0x7f30e013f670 legacy=0x7f30e00b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c8003c70 0x7f30c8004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c8004810 0x7f30c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8005210 0x7f30c800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e006f0e0 con 0x7f30c8003c70 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c800c130 con 0x7f30c8004810 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c800c2f0 con 0x7f30c8005210 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c8003c70 0x7f30c8004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5402700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c8004810 0x7f30c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6404700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8005210 0x7f30c800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 -- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8005210 msgr2=0x7f30c800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8005210 0x7f30c800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 -- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c8004810 msgr2=0x7f30c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c8004810 0x7f30c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 -- 172.21.15.100:0/261862552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8003570 con 0x7f30c8003c70 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6404700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8005210 0x7f30c800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e6404700 1 -- 172.21.15.100:0/261862552 reap_dead start 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e5c03700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c8003c70 0x7f30c8004060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30d4007fd0 tx=0x7f30d4001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1aec5670bc6a43 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400cae0 con 0x7f30c8003c70 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d400cc40 con 0x7f30c8003c70 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.576+0000 7f30e4c01700 1 -- 172.21.15.100:0/261862552 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4005920 con 0x7f30c8003c70 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 -- 172.21.15.100:0/261862552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c8003c70 msgr2=0x7f30c8004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c8003c70 0x7f30c8004060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30d4007fd0 tx=0x7f30d4001d40 comp rx=0 tx=0).stop 2024-08-17T02:22:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 -- 172.21.15.100:0/261862552 shutdown_connections 2024-08-17T02:22:08.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 --2- 172.21.15.100:0/261862552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c8003c70 0x7f30c8004060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 -- 172.21.15.100:0/261862552 >> 172.21.15.100:0/261862552 conn(0x7f30e00ca2b0 msgr2=0x7f30e006e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:08.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 -- 172.21.15.100:0/261862552 shutdown_connections 2024-08-17T02:22:08.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 -- 172.21.15.100:0/261862552 wait complete. 2024-08-17T02:22:08.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 Processor -- start 2024-08-17T02:22:08.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.580+0000 7f30e6c05700 1 -- start start 2024-08-17T02:22:08.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8003c70 0x7f30e0155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e006a7e0 0x7f30e006ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30e00ab590 0x7f30e00ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e006b190 con 0x7f30e006a7e0 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00a6a90 con 0x7f30e00ab590 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00ac260 con 0x7f30c8003c70 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e006a7e0 0x7f30e006ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e006a7e0 0x7f30e006ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56720/0 (socket says 172.21.15.100:56720) 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 -- 172.21.15.100:0/2473695993 learned_addr learned my addr 172.21.15.100:0/2473695993 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5c03700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8003c70 0x7f30e0155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6404700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30e00ab590 0x7f30e00ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 -- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8003c70 msgr2=0x7f30e0155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8003c70 0x7f30e0155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 -- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30e00ab590 msgr2=0x7f30e00ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30e00ab590 0x7f30e00ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 -- 172.21.15.100:0/2473695993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00a6160 con 0x7f30e006a7e0 2024-08-17T02:22:08.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e5402700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e006a7e0 0x7f30e006ac50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f30d000abb0 tx=0x7f30d00017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30ddffb700 1 -- 172.21.15.100:0/2473695993 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0006e80 con 0x7f30e006a7e0 2024-08-17T02:22:08.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30ddffb700 1 -- 172.21.15.100:0/2473695993 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0010670 con 0x7f30e006a7e0 2024-08-17T02:22:08.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e00a6440 con 0x7f30e006a7e0 2024-08-17T02:22:08.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30ddffb700 1 -- 172.21.15.100:0/2473695993 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0010a30 con 0x7f30e006a7e0 2024-08-17T02:22:08.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e00abf20 con 0x7f30e006a7e0 2024-08-17T02:22:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.584+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e00a6c40 con 0x7f30e006a7e0 2024-08-17T02:22:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.588+0000 7f30ddffb700 1 -- 172.21.15.100:0/2473695993 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f30d0014400 con 0x7f30e006a7e0 2024-08-17T02:22:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.588+0000 7f30ddffb700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f30cc060890 0x7f30cc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.588+0000 7f30ddffb700 1 -- 172.21.15.100:0/2473695993 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f30d007f470 con 0x7f30e006a7e0 2024-08-17T02:22:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.588+0000 7f30e5c03700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f30cc060890 0x7f30cc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.588+0000 7f30e5c03700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f30cc060890 0x7f30cc062d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30d4001120 tx=0x7f30d4015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.596+0000 7f30ddffb700 1 -- 172.21.15.100:0/2473695993 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f30e00a6c40 con 0x7f30e006a7e0 2024-08-17T02:22:08.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:22:08.665 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:22:08.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.756+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30e00a6c00 con 0x7f30cc060890 2024-08-17T02:22:08.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.772+0000 7f30ddffb700 1 -- 172.21.15.100:0/2473695993 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252213 (secure 0 0 0) 0x7f30e00a6c00 con 0x7f30cc060890 2024-08-17T02:22:08.777 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:08.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f30cc060890 msgr2=0x7f30cc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f30cc060890 0x7f30cc062d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30d4001120 tx=0x7f30d4015000 comp rx=0 tx=0).stop 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e006a7e0 msgr2=0x7f30e006ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e006a7e0 0x7f30e006ac50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f30d000abb0 tx=0x7f30d00017a0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 shutdown_connections 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f30cc060890 0x7f30cc062d40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30e00ab590 0x7f30e00ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e006a7e0 0x7f30e006ac50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 --2- 172.21.15.100:0/2473695993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c8003c70 0x7f30e0155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 >> 172.21.15.100:0/2473695993 conn(0x7f30e00ca2b0 msgr2=0x7f30e00ac780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 shutdown_connections 2024-08-17T02:22:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.776+0000 7f30e6c05700 1 -- 172.21.15.100:0/2473695993 wait complete. 2024-08-17T02:22:08.781 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:08.792 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-08-17T02:22:07.221728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381784,"kb_used_data":2536,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592936,"statfs":{"total":773094113280,"available":770655166464,"internally_reserved":0,"allocated":2596864,"data_stored":1607946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.120334"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.099961+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100907+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100678+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095631+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:28:53.644529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101508+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:07:22.383055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096031+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103503+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:07:22.383055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100784+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100245+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:50.168021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.097451+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100709+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095296+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.094655+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095796+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101956+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096039+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":160,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-17T02:22:02.815000+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:02.815000+0000","last_peered":"2024-08-17T02:22:02.815000+0000","last_clean":"2024-08-17T02:22:02.815000+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:02.815000+0000","last_undegraded":"2024-08-17T02:22:02.815000+0000","last_fullsized":"2024-08-17T02:22:02.815000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.095996+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:11.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.104598+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096212+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.096355+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100878+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103730+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101835+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100653+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.105400+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:48:47.160779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.100688+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101002+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103972+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.103678+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.104331+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101123+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:22:04.089363+0000","last_change":"2024-08-17T02:22:04.089363+0000","last_active":"2024-08-17T02:22:04.089363+0000","last_peered":"2024-08-17T02:22:04.089363+0000","last_clean":"2024-08-17T02:22:04.089363+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:04.089363+0000","last_undegraded":"2024-08-17T02:22:04.089363+0000","last_fullsized":"2024-08-17T02:22:04.089363+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:22:03.085579+0000","last_change":"2024-08-17T02:22:03.101303+0000","last_active":"2024-08-17T02:22:03.085579+0000","last_peered":"2024-08-17T02:22:03.085579+0000","last_clean":"2024-08-17T02:22:03.085579+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:22:03.085579+0000","last_undegraded":"2024-08-17T02:22:03.085579+0000","last_fullsized":"2024-08-17T02:22:03.085579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:28:53.644529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298308,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073532,"statfs":{"total":96636764160,"available":96331296768,"internally_reserved":0,"allocated":782336,"data_stored":649428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.831}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]}]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298308,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073532,"statfs":{"total":96636764160,"available":96331296768,"internally_reserved":0,"allocated":782336,"data_stored":649428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:08.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-08-17T02:22:08.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-08-17T02:22:08.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-08-17T02:22:08.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-08-17T02:22:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-08-17T02:22:08.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-08-17T02:22:08.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-08-17T02:22:08.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.099961+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-17T02:22:08.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-08-17T02:22:08.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.100907+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.100678+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.095631+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T07:28:53.644529+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-08-17T02:22:08.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.101508+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T07:07:22.383055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.096031+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-08-17T02:22:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.103503+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T07:07:22.383055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.100784+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.100245+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T04:33:50.168021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.097451+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.100709+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.095296+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-17T02:22:08.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.094655+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.095796+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.101956+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.096039+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.095996+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T09:34:11.373621+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-08-17T02:22:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-08-17T02:22:08.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.104598+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-17T02:22:08.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-08-17T02:22:08.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-08-17T02:22:08.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.096212+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-17T02:22:08.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.096355+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.100878+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.103730+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T06:13:40.191338+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.101835+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.100653+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-08-17T02:22:08.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-08-17T02:22:08.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.105400+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T06:48:47.160779+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-17T02:22:08.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-08-17T02:22:08.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-08-17T02:22:08.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.100688+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-17T02:22:08.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-08-17T02:22:08.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.101002+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T13:34:29.446500+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.103972+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.103678+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T07:38:15.424101+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.104331+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.101123+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T05:24:40.839265+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-08-17T02:22:08.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-17T02:22:04.089363+0000', 'last_change': '2024-08-17T02:22:04.089363+0000', 'last_active': '2024-08-17T02:22:04.089363+0000', 'last_peered': '2024-08-17T02:22:04.089363+0000', 'last_clean': '2024-08-17T02:22:04.089363+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:04.089363+0000', 'last_undegraded': '2024-08-17T02:22:04.089363+0000', 'last_fullsized': '2024-08-17T02:22:04.089363+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:04.089363+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-17T02:22:08.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-08-17T02:22:08.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-17T02:22:03.085579+0000', 'last_change': '2024-08-17T02:22:03.101303+0000', 'last_active': '2024-08-17T02:22:03.085579+0000', 'last_peered': '2024-08-17T02:22:03.085579+0000', 'last_clean': '2024-08-17T02:22:03.085579+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-17T02:22:03.085579+0000', 'last_undegraded': '2024-08-17T02:22:03.085579+0000', 'last_fullsized': '2024-08-17T02:22:03.085579+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'last_clean_scrub_stamp': '2024-08-17T02:22:03.085579+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-18T07:28:53.644529+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-17T02:22:08.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-08-17T02:22:08.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.920+0000 7f75563a4700 1 Processor -- start 2024-08-17T02:22:08.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.920+0000 7f75563a4700 1 -- start start 2024-08-17T02:22:08.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75500af410 0x7f75500ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7550155fb0 0x7f75500acde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75500ad650 con 0x7f7550155be0 2024-08-17T02:22:08.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75500ad930 con 0x7f7550155fb0 2024-08-17T02:22:08.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75500adc10 con 0x7f75500af410 2024-08-17T02:22:08.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7555ba3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7550155fb0 0x7f75500acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7555ba3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7550155fb0 0x7f75500acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56722/0 (socket says 172.21.15.100:56722) 2024-08-17T02:22:08.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7555ba3700 1 -- 172.21.15.100:0/2392925444 learned_addr learned my addr 172.21.15.100:0/2392925444 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:08.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7554ba1700 1 --2- 172.21.15.100:0/2392925444 >> v2:172.21.15.123:3300/0 conn(0x7f75500af410 0x7f75500ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7555ba3700 1 -- 172.21.15.100:0/2392925444 >> v2:172.21.15.123:3300/0 conn(0x7f75500af410 msgr2=0x7f75500ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:08.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7555ba3700 1 --2- 172.21.15.100:0/2392925444 >> v2:172.21.15.123:3300/0 conn(0x7f75500af410 0x7f75500ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7555ba3700 1 -- 172.21.15.100:0/2392925444 >> v1:172.21.15.123:6789/0 conn(0x7f7550155be0 legacy=0x7f75500aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7555ba3700 1 -- 172.21.15.100:0/2392925444 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500adef0 con 0x7f7550155fb0 2024-08-17T02:22:08.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7555ba3700 1 --2- 172.21.15.100:0/2392925444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500acde0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f755006d450 tx=0x7f7548009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea6079051e09624 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7547fff700 1 -- 172.21.15.100:0/2392925444 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800e070 con 0x7f7550155fb0 2024-08-17T02:22:08.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7547fff700 1 -- 172.21.15.100:0/2392925444 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754800ba20 con 0x7f7550155fb0 2024-08-17T02:22:08.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f7547fff700 1 -- 172.21.15.100:0/2392925444 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75480092e0 con 0x7f7550155fb0 2024-08-17T02:22:08.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 -- 172.21.15.100:0/2392925444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 msgr2=0x7f75500acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 --2- 172.21.15.100:0/2392925444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500acde0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f755006d450 tx=0x7f7548009840 comp rx=0 tx=0).stop 2024-08-17T02:22:08.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 -- 172.21.15.100:0/2392925444 shutdown_connections 2024-08-17T02:22:08.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 --2- 172.21.15.100:0/2392925444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500acde0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 --2- 172.21.15.100:0/2392925444 >> v2:172.21.15.123:3300/0 conn(0x7f75500af410 0x7f75500ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 -- 172.21.15.100:0/2392925444 >> 172.21.15.100:0/2392925444 conn(0x7f755006e070 msgr2=0x7f75500b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:08.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 -- 172.21.15.100:0/2392925444 shutdown_connections 2024-08-17T02:22:08.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.924+0000 7f75563a4700 1 -- 172.21.15.100:0/2392925444 wait complete. 2024-08-17T02:22:08.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75563a4700 1 Processor -- start 2024-08-17T02:22:08.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75563a4700 1 -- start start 2024-08-17T02:22:08.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75563a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75500af410 0x7f75501439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75563a4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7550155be0 0x7f75500afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75563a4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75563a4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7550068ad0 con 0x7f7550155fb0 2024-08-17T02:22:08.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75563a4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75500b0580 con 0x7f75500af410 2024-08-17T02:22:08.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75563a4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75500b0860 con 0x7f7550155be0 2024-08-17T02:22:08.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56726/0 (socket says 172.21.15.100:56726) 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 -- 172.21.15.100:0/1124934469 learned_addr learned my addr 172.21.15.100:0/1124934469 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7554ba1700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7550155be0 0x7f75500afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75553a2700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75500af410 0x7f75501439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 -- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7550155be0 msgr2=0x7f75500afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7550155be0 0x7f75500afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 -- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75500af410 msgr2=0x7f75501439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75500af410 0x7f75501439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 -- 172.21.15.100:0/1124934469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7548009ae0 con 0x7f7550155fb0 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f75553a2700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75500af410 0x7f75501439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.928+0000 7f7555ba3700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500b1ce0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f75500b3010 tx=0x7f7548018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75457fa700 1 -- 172.21.15.100:0/1124934469 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754801f070 con 0x7f7550155fb0 2024-08-17T02:22:08.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75457fa700 1 -- 172.21.15.100:0/1124934469 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548008ab0 con 0x7f7550155fb0 2024-08-17T02:22:08.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75500b0b40 con 0x7f7550155fb0 2024-08-17T02:22:08.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75457fa700 1 -- 172.21.15.100:0/1124934469 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548008e70 con 0x7f7550155fb0 2024-08-17T02:22:08.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7550166340 con 0x7f7550155fb0 2024-08-17T02:22:08.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7550068c80 con 0x7f7550155fb0 2024-08-17T02:22:08.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75457fa700 1 -- 172.21.15.100:0/1124934469 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7548003040 con 0x7f7550155fb0 2024-08-17T02:22:08.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75457fa700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7534060860 0x7f7534062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:08.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75457fa700 1 -- 172.21.15.100:0/1124934469 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f754807fd90 con 0x7f7550155fb0 2024-08-17T02:22:08.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75553a2700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7534060860 0x7f7534062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:08.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.932+0000 7f75553a2700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7534060860 0x7f7534062d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f754c007fd0 tx=0x7f754c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:08.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:08.940+0000 7f75457fa700 1 -- 172.21.15.100:0/1124934469 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7550068c80 con 0x7f7550155fb0 2024-08-17T02:22:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.160+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f75500638f0 con 0x7f7550155fb0 2024-08-17T02:22:09.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75457fa700 1 -- 172.21.15.100:0/1124934469 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1109 (secure 0 0 0) 0x7f75480501e0 con 0x7f7550155fb0 2024-08-17T02:22:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:09.165 INFO:teuthology.orchestra.run.smithi100.stdout:{"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":22,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1723861317,"num_in_osds":8,"osd_in_since":1723861309,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2438946816,"bytes_avail":770655166464,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4270}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-08-17T02:21:46.997155+0000","services":{}},"progress_events":{}} 2024-08-17T02:22:09.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7534060860 msgr2=0x7f7534062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7534060860 0x7f7534062d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f754c007fd0 tx=0x7f754c009040 comp rx=0 tx=0).stop 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 msgr2=0x7f75500b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500b1ce0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f75500b3010 tx=0x7f7548018000 comp rx=0 tx=0).stop 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 shutdown_connections 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7534060860 0x7f7534062d10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550155fb0 0x7f75500b1ce0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7550155be0 0x7f75500afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 --2- 172.21.15.100:0/1124934469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75500af410 0x7f75501439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 >> 172.21.15.100:0/1124934469 conn(0x7f755006e070 msgr2=0x7f75501549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 shutdown_connections 2024-08-17T02:22:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.164+0000 7f75563a4700 1 -- 172.21.15.100:0/1124934469 wait complete. 2024-08-17T02:22:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-08-17T02:22:09.185 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:09.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.292+0000 7f469c936700 1 Processor -- start 2024-08-17T02:22:09.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.292+0000 7f469c936700 1 -- start start 2024-08-17T02:22:09.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f469c936700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4698155390 0x7f4698155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:09.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f469c936700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469806b6d0 con 0x7f46980682a0 2024-08-17T02:22:09.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f469c936700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469806b3f0 con 0x7f46980c9dd0 2024-08-17T02:22:09.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4696d9d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4698155390 0x7f4698155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:09.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f469c936700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4698068700 con 0x7f4698155390 2024-08-17T02:22:09.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f469659c700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f46980682a0 0x7f46980c96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46972/0 (socket says 172.21.15.100:46972) 2024-08-17T02:22:09.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f469659c700 1 -- 172.21.15.100:0/2998161938 learned_addr learned my addr 172.21.15.100:0/2998161938 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:09.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4695d9b700 1 -- 172.21.15.100:0/2998161938 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291827900 0 0) 0x7f469806b6d0 con 0x7f46980682a0 2024-08-17T02:22:09.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4695d9b700 1 -- 172.21.15.100:0/2998161938 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4680003570 con 0x7f46980682a0 2024-08-17T02:22:09.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4696d9d700 1 -- 172.21.15.100:0/2998161938 >> v1:172.21.15.123:6789/0 conn(0x7f46980c9dd0 legacy=0x7f46980697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4696d9d700 1 -- 172.21.15.100:0/2998161938 >> v1:172.21.15.100:6789/0 conn(0x7f46980682a0 legacy=0x7f46980c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4696d9d700 1 -- 172.21.15.100:0/2998161938 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980677c0 con 0x7f4698155390 2024-08-17T02:22:09.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4696d9d700 1 --2- 172.21.15.100:0/2998161938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f4698155760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f46880098e0 tx=0x7f46880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a17352b8aec5624 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:09.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4695d9b700 1 -- 172.21.15.100:0/2998161938 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468800b640 con 0x7f4698155390 2024-08-17T02:22:09.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4695d9b700 1 -- 172.21.15.100:0/2998161938 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468800b7a0 con 0x7f4698155390 2024-08-17T02:22:09.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f4695d9b700 1 -- 172.21.15.100:0/2998161938 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468800bb40 con 0x7f4698155390 2024-08-17T02:22:09.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.296+0000 7f469c936700 1 -- 172.21.15.100:0/2998161938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 msgr2=0x7f4698155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.300+0000 7f469c936700 1 --2- 172.21.15.100:0/2998161938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f4698155760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f46880098e0 tx=0x7f46880091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.300+0000 7f469c936700 1 -- 172.21.15.100:0/2998161938 shutdown_connections 2024-08-17T02:22:09.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.300+0000 7f469c936700 1 --2- 172.21.15.100:0/2998161938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f4698155760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.300+0000 7f469c936700 1 -- 172.21.15.100:0/2998161938 >> 172.21.15.100:0/2998161938 conn(0x7f46980ccfc0 msgr2=0x7f46980c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:09.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.300+0000 7f469c936700 1 -- 172.21.15.100:0/2998161938 shutdown_connections 2024-08-17T02:22:09.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.300+0000 7f469c936700 1 -- 172.21.15.100:0/2998161938 wait complete. 2024-08-17T02:22:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.300+0000 7f469c936700 1 Processor -- start 2024-08-17T02:22:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.300+0000 7f469c936700 1 -- start start 2024-08-17T02:22:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46980682a0 0x7f46981450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980c9dd0 0x7f46980c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f469806a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980a5450 con 0x7f4698155390 2024-08-17T02:22:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980a5fd0 con 0x7f46980c9dd0 2024-08-17T02:22:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46980a56a0 con 0x7f46980682a0 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f469806a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f469806a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56734/0 (socket says 172.21.15.100:56734) 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980c9dd0 0x7f46980c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 -- 172.21.15.100:0/1811867798 learned_addr learned my addr 172.21.15.100:0/1811867798 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f4696d9d700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46980682a0 0x7f46981450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 -- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46980682a0 msgr2=0x7f46981450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46980682a0 0x7f46981450b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 -- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980c9dd0 msgr2=0x7f46980c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980c9dd0 0x7f46980c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 -- 172.21.15.100:0/1811867798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688009420 con 0x7f4698155390 2024-08-17T02:22:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469759e700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f469806a500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f469000c870 tx=0x7f4690001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f467effd700 1 -- 172.21.15.100:0/1811867798 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690012630 con 0x7f4698155390 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f467effd700 1 -- 172.21.15.100:0/1811867798 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46900127b0 con 0x7f4698155390 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46980a5980 con 0x7f4698155390 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f467effd700 1 -- 172.21.15.100:0/1811867798 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690012b70 con 0x7f4698155390 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46981455f0 con 0x7f4698155390 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46980638f0 con 0x7f4698155390 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.304+0000 7f467effd700 1 -- 172.21.15.100:0/1811867798 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4690003030 con 0x7f4698155390 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.308+0000 7f467effd700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4684060860 0x7f4684062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.308+0000 7f467effd700 1 -- 172.21.15.100:0/1811867798 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f469007ca50 con 0x7f4698155390 2024-08-17T02:22:09.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.308+0000 7f4696d9d700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4684060860 0x7f4684062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:09.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.308+0000 7f4696d9d700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4684060860 0x7f4684062d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46880093f0 tx=0x7f4688017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:09.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.316+0000 7f467effd700 1 -- 172.21.15.100:0/1811867798 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f469004cea0 con 0x7f4698155390 2024-08-17T02:22:09.450 INFO:tasks.ceph.mds.d.smithi123.stderr:2024-08-17T02:22:09.446+0000 7f8f96042700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:22:09.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.476+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46980a6110 con 0x7f4684060860 2024-08-17T02:22:09.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.496+0000 7f467effd700 1 -- 172.21.15.100:0/1811867798 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264496 (secure 0 0 0) 0x7f46980a6110 con 0x7f4684060860 2024-08-17T02:22:09.501 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4684060860 msgr2=0x7f4684062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4684060860 0x7f4684062d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46880093f0 tx=0x7f4688017000 comp rx=0 tx=0).stop 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 msgr2=0x7f469806a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f469806a500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f469000c870 tx=0x7f4690001670 comp rx=0 tx=0).stop 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 shutdown_connections 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4684060860 0x7f4684062d10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4698155390 0x7f469806a500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980c9dd0 0x7f46980c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 --2- 172.21.15.100:0/1811867798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46980682a0 0x7f46981450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 >> 172.21.15.100:0/1811867798 conn(0x7f46980ccfc0 msgr2=0x7f46980b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 shutdown_connections 2024-08-17T02:22:09.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:09.500+0000 7f469c936700 1 -- 172.21.15.100:0/1811867798 wait complete. 2024-08-17T02:22:09.508 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:09.520 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-08-17T02:22:09.222381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382032,"kb_used_data":2960,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592688,"statfs":{"total":773094113280,"available":770654912512,"internally_reserved":0,"allocated":3031040,"data_stored":1799828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.120987"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143110+0000","last_change":"2024-08-17T02:22:05.137118+0000","last_active":"2024-08-17T02:22:05.143110+0000","last_peered":"2024-08-17T02:22:05.143110+0000","last_clean":"2024-08-17T02:22:05.143110+0000","last_became_active":"2024-08-17T02:22:05.136784+0000","last_became_peered":"2024-08-17T02:22:05.136784+0000","last_unstale":"2024-08-17T02:22:05.143110+0000","last_undegraded":"2024-08-17T02:22:05.143110+0000","last_fullsized":"2024-08-17T02:22:05.143110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:04:25.260837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133391+0000","last_change":"2024-08-17T02:22:05.129636+0000","last_active":"2024-08-17T02:22:05.133391+0000","last_peered":"2024-08-17T02:22:05.133391+0000","last_clean":"2024-08-17T02:22:05.133391+0000","last_became_active":"2024-08-17T02:22:05.129342+0000","last_became_peered":"2024-08-17T02:22:05.129342+0000","last_unstale":"2024-08-17T02:22:05.133391+0000","last_undegraded":"2024-08-17T02:22:05.133391+0000","last_fullsized":"2024-08-17T02:22:05.133391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.131713+0000","last_change":"2024-08-17T02:22:05.128097+0000","last_active":"2024-08-17T02:22:05.131713+0000","last_peered":"2024-08-17T02:22:05.131713+0000","last_clean":"2024-08-17T02:22:05.131713+0000","last_became_active":"2024-08-17T02:22:05.127786+0000","last_became_peered":"2024-08-17T02:22:05.127786+0000","last_unstale":"2024-08-17T02:22:05.131713+0000","last_undegraded":"2024-08-17T02:22:05.131713+0000","last_fullsized":"2024-08-17T02:22:05.131713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:47.605286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142739+0000","last_change":"2024-08-17T02:22:05.138911+0000","last_active":"2024-08-17T02:22:05.142739+0000","last_peered":"2024-08-17T02:22:05.142739+0000","last_clean":"2024-08-17T02:22:05.142739+0000","last_became_active":"2024-08-17T02:22:05.134886+0000","last_became_peered":"2024-08-17T02:22:05.134886+0000","last_unstale":"2024-08-17T02:22:05.142739+0000","last_undegraded":"2024-08-17T02:22:05.142739+0000","last_fullsized":"2024-08-17T02:22:05.142739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:04:25.260837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143121+0000","last_change":"2024-08-17T02:22:05.137212+0000","last_active":"2024-08-17T02:22:05.143121+0000","last_peered":"2024-08-17T02:22:05.143121+0000","last_clean":"2024-08-17T02:22:05.143121+0000","last_became_active":"2024-08-17T02:22:05.136566+0000","last_became_peered":"2024-08-17T02:22:05.136566+0000","last_unstale":"2024-08-17T02:22:05.143121+0000","last_undegraded":"2024-08-17T02:22:05.143121+0000","last_fullsized":"2024-08-17T02:22:05.143121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:49.230407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141537+0000","last_change":"2024-08-17T02:22:05.130974+0000","last_active":"2024-08-17T02:22:05.141537+0000","last_peered":"2024-08-17T02:22:05.141537+0000","last_clean":"2024-08-17T02:22:05.141537+0000","last_became_active":"2024-08-17T02:22:05.130696+0000","last_became_peered":"2024-08-17T02:22:05.130696+0000","last_unstale":"2024-08-17T02:22:05.141537+0000","last_undegraded":"2024-08-17T02:22:05.141537+0000","last_fullsized":"2024-08-17T02:22:05.141537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:49.230407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143027+0000","last_change":"2024-08-17T02:22:05.137564+0000","last_active":"2024-08-17T02:22:05.143027+0000","last_peered":"2024-08-17T02:22:05.143027+0000","last_clean":"2024-08-17T02:22:05.143027+0000","last_became_active":"2024-08-17T02:22:05.137060+0000","last_became_peered":"2024-08-17T02:22:05.137060+0000","last_unstale":"2024-08-17T02:22:05.143027+0000","last_undegraded":"2024-08-17T02:22:05.143027+0000","last_fullsized":"2024-08-17T02:22:05.143027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:48:26.336166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146835+0000","last_change":"2024-08-17T02:22:05.143886+0000","last_active":"2024-08-17T02:22:05.146835+0000","last_peered":"2024-08-17T02:22:05.146835+0000","last_clean":"2024-08-17T02:22:05.146835+0000","last_became_active":"2024-08-17T02:22:05.143537+0000","last_became_peered":"2024-08-17T02:22:05.143537+0000","last_unstale":"2024-08-17T02:22:05.146835+0000","last_undegraded":"2024-08-17T02:22:05.146835+0000","last_fullsized":"2024-08-17T02:22:05.146835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:58:32.777645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146777+0000","last_change":"2024-08-17T02:22:05.142931+0000","last_active":"2024-08-17T02:22:05.146777+0000","last_peered":"2024-08-17T02:22:05.146777+0000","last_clean":"2024-08-17T02:22:05.146777+0000","last_became_active":"2024-08-17T02:22:05.142076+0000","last_became_peered":"2024-08-17T02:22:05.142076+0000","last_unstale":"2024-08-17T02:22:05.146777+0000","last_undegraded":"2024-08-17T02:22:05.146777+0000","last_fullsized":"2024-08-17T02:22:05.146777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:54:13.258567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133864+0000","last_change":"2024-08-17T02:22:05.131490+0000","last_active":"2024-08-17T02:22:05.133864+0000","last_peered":"2024-08-17T02:22:05.133864+0000","last_clean":"2024-08-17T02:22:05.133864+0000","last_became_active":"2024-08-17T02:22:05.131184+0000","last_became_peered":"2024-08-17T02:22:05.131184+0000","last_unstale":"2024-08-17T02:22:05.133864+0000","last_undegraded":"2024-08-17T02:22:05.133864+0000","last_fullsized":"2024-08-17T02:22:05.133864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134444+0000","last_change":"2024-08-17T02:22:05.131423+0000","last_active":"2024-08-17T02:22:05.134444+0000","last_peered":"2024-08-17T02:22:05.134444+0000","last_clean":"2024-08-17T02:22:05.134444+0000","last_became_active":"2024-08-17T02:22:05.131081+0000","last_became_peered":"2024-08-17T02:22:05.131081+0000","last_unstale":"2024-08-17T02:22:05.134444+0000","last_undegraded":"2024-08-17T02:22:05.134444+0000","last_fullsized":"2024-08-17T02:22:05.134444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133327+0000","last_change":"2024-08-17T02:22:05.129487+0000","last_active":"2024-08-17T02:22:05.133327+0000","last_peered":"2024-08-17T02:22:05.133327+0000","last_clean":"2024-08-17T02:22:05.133327+0000","last_became_active":"2024-08-17T02:22:05.129222+0000","last_became_peered":"2024-08-17T02:22:05.129222+0000","last_unstale":"2024-08-17T02:22:05.133327+0000","last_undegraded":"2024-08-17T02:22:05.133327+0000","last_fullsized":"2024-08-17T02:22:05.133327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:02.706489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138694+0000","last_change":"2024-08-17T02:22:05.134693+0000","last_active":"2024-08-17T02:22:05.138694+0000","last_peered":"2024-08-17T02:22:05.138694+0000","last_clean":"2024-08-17T02:22:05.138694+0000","last_became_active":"2024-08-17T02:22:05.134300+0000","last_became_peered":"2024-08-17T02:22:05.134300+0000","last_unstale":"2024-08-17T02:22:05.138694+0000","last_undegraded":"2024-08-17T02:22:05.138694+0000","last_fullsized":"2024-08-17T02:22:05.138694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:37.064590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.131330+0000","last_change":"2024-08-17T02:22:05.128086+0000","last_active":"2024-08-17T02:22:05.131330+0000","last_peered":"2024-08-17T02:22:05.131330+0000","last_clean":"2024-08-17T02:22:05.131330+0000","last_became_active":"2024-08-17T02:22:05.127750+0000","last_became_peered":"2024-08-17T02:22:05.127750+0000","last_unstale":"2024-08-17T02:22:05.131330+0000","last_undegraded":"2024-08-17T02:22:05.131330+0000","last_fullsized":"2024-08-17T02:22:05.131330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:01:27.557725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.123798+0000","last_change":"2024-08-17T02:22:05.123028+0000","last_active":"2024-08-17T02:22:05.123798+0000","last_peered":"2024-08-17T02:22:05.123798+0000","last_clean":"2024-08-17T02:22:05.123798+0000","last_became_active":"2024-08-17T02:22:05.122711+0000","last_became_peered":"2024-08-17T02:22:05.122711+0000","last_unstale":"2024-08-17T02:22:05.123798+0000","last_undegraded":"2024-08-17T02:22:05.123798+0000","last_fullsized":"2024-08-17T02:22:05.123798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141452+0000","last_change":"2024-08-17T02:22:05.130380+0000","last_active":"2024-08-17T02:22:05.141452+0000","last_peered":"2024-08-17T02:22:05.141452+0000","last_clean":"2024-08-17T02:22:05.141452+0000","last_became_active":"2024-08-17T02:22:05.130077+0000","last_became_peered":"2024-08-17T02:22:05.130077+0000","last_unstale":"2024-08-17T02:22:05.141452+0000","last_undegraded":"2024-08-17T02:22:05.141452+0000","last_fullsized":"2024-08-17T02:22:05.141452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:27:57.075296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142733+0000","last_change":"2024-08-17T02:22:05.139113+0000","last_active":"2024-08-17T02:22:05.142733+0000","last_peered":"2024-08-17T02:22:05.142733+0000","last_clean":"2024-08-17T02:22:05.142733+0000","last_became_active":"2024-08-17T02:22:05.138752+0000","last_became_peered":"2024-08-17T02:22:05.138752+0000","last_unstale":"2024-08-17T02:22:05.142733+0000","last_undegraded":"2024-08-17T02:22:05.142733+0000","last_fullsized":"2024-08-17T02:22:05.142733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:21:52.626562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.129913+0000","last_change":"2024-08-17T02:22:05.125930+0000","last_active":"2024-08-17T02:22:05.129913+0000","last_peered":"2024-08-17T02:22:05.129913+0000","last_clean":"2024-08-17T02:22:05.129913+0000","last_became_active":"2024-08-17T02:22:05.125640+0000","last_became_peered":"2024-08-17T02:22:05.125640+0000","last_unstale":"2024-08-17T02:22:05.129913+0000","last_undegraded":"2024-08-17T02:22:05.129913+0000","last_fullsized":"2024-08-17T02:22:05.129913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:59:54.522675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142959+0000","last_change":"2024-08-17T02:22:05.137818+0000","last_active":"2024-08-17T02:22:05.142959+0000","last_peered":"2024-08-17T02:22:05.142959+0000","last_clean":"2024-08-17T02:22:05.142959+0000","last_became_active":"2024-08-17T02:22:05.137531+0000","last_became_peered":"2024-08-17T02:22:05.137531+0000","last_unstale":"2024-08-17T02:22:05.142959+0000","last_undegraded":"2024-08-17T02:22:05.142959+0000","last_fullsized":"2024-08-17T02:22:05.142959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:44:40.942386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.137413+0000","last_change":"2024-08-17T02:22:05.135163+0000","last_active":"2024-08-17T02:22:05.137413+0000","last_peered":"2024-08-17T02:22:05.137413+0000","last_clean":"2024-08-17T02:22:05.137413+0000","last_became_active":"2024-08-17T02:22:05.134647+0000","last_became_peered":"2024-08-17T02:22:05.134647+0000","last_unstale":"2024-08-17T02:22:05.137413+0000","last_undegraded":"2024-08-17T02:22:05.137413+0000","last_fullsized":"2024-08-17T02:22:05.137413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:21.690535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144205+0000","last_change":"2024-08-17T02:22:05.137440+0000","last_active":"2024-08-17T02:22:05.144205+0000","last_peered":"2024-08-17T02:22:05.144205+0000","last_clean":"2024-08-17T02:22:05.144205+0000","last_became_active":"2024-08-17T02:22:05.137081+0000","last_became_peered":"2024-08-17T02:22:05.137081+0000","last_unstale":"2024-08-17T02:22:05.144205+0000","last_undegraded":"2024-08-17T02:22:05.144205+0000","last_fullsized":"2024-08-17T02:22:05.144205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:34:43.102929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142978+0000","last_change":"2024-08-17T02:22:05.139220+0000","last_active":"2024-08-17T02:22:05.142978+0000","last_peered":"2024-08-17T02:22:05.142978+0000","last_clean":"2024-08-17T02:22:05.142978+0000","last_became_active":"2024-08-17T02:22:05.138838+0000","last_became_peered":"2024-08-17T02:22:05.138838+0000","last_unstale":"2024-08-17T02:22:05.142978+0000","last_undegraded":"2024-08-17T02:22:05.142978+0000","last_fullsized":"2024-08-17T02:22:05.142978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:47:26.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135504+0000","last_change":"2024-08-17T02:22:05.132463+0000","last_active":"2024-08-17T02:22:05.135504+0000","last_peered":"2024-08-17T02:22:05.135504+0000","last_clean":"2024-08-17T02:22:05.135504+0000","last_became_active":"2024-08-17T02:22:05.131741+0000","last_became_peered":"2024-08-17T02:22:05.131741+0000","last_unstale":"2024-08-17T02:22:05.135504+0000","last_undegraded":"2024-08-17T02:22:05.135504+0000","last_fullsized":"2024-08-17T02:22:05.135504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142777+0000","last_change":"2024-08-17T02:22:05.140510+0000","last_active":"2024-08-17T02:22:05.142777+0000","last_peered":"2024-08-17T02:22:05.142777+0000","last_clean":"2024-08-17T02:22:05.142777+0000","last_became_active":"2024-08-17T02:22:05.139145+0000","last_became_peered":"2024-08-17T02:22:05.139145+0000","last_unstale":"2024-08-17T02:22:05.142777+0000","last_undegraded":"2024-08-17T02:22:05.142777+0000","last_fullsized":"2024-08-17T02:22:05.142777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:58:53.092915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135524+0000","last_change":"2024-08-17T02:22:05.133736+0000","last_active":"2024-08-17T02:22:05.135524+0000","last_peered":"2024-08-17T02:22:05.135524+0000","last_clean":"2024-08-17T02:22:05.135524+0000","last_became_active":"2024-08-17T02:22:05.132607+0000","last_became_peered":"2024-08-17T02:22:05.132607+0000","last_unstale":"2024-08-17T02:22:05.135524+0000","last_undegraded":"2024-08-17T02:22:05.135524+0000","last_fullsized":"2024-08-17T02:22:05.135524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141845+0000","last_change":"2024-08-17T02:22:05.137747+0000","last_active":"2024-08-17T02:22:05.141845+0000","last_peered":"2024-08-17T02:22:05.141845+0000","last_clean":"2024-08-17T02:22:05.141845+0000","last_became_active":"2024-08-17T02:22:05.137464+0000","last_became_peered":"2024-08-17T02:22:05.137464+0000","last_unstale":"2024-08-17T02:22:05.141845+0000","last_undegraded":"2024-08-17T02:22:05.141845+0000","last_fullsized":"2024-08-17T02:22:05.141845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138959+0000","last_change":"2024-08-17T02:22:05.133235+0000","last_active":"2024-08-17T02:22:05.138959+0000","last_peered":"2024-08-17T02:22:05.138959+0000","last_clean":"2024-08-17T02:22:05.138959+0000","last_became_active":"2024-08-17T02:22:05.132962+0000","last_became_peered":"2024-08-17T02:22:05.132962+0000","last_unstale":"2024-08-17T02:22:05.138959+0000","last_undegraded":"2024-08-17T02:22:05.138959+0000","last_fullsized":"2024-08-17T02:22:05.138959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135434+0000","last_change":"2024-08-17T02:22:05.132315+0000","last_active":"2024-08-17T02:22:05.135434+0000","last_peered":"2024-08-17T02:22:05.135434+0000","last_clean":"2024-08-17T02:22:05.135434+0000","last_became_active":"2024-08-17T02:22:05.131822+0000","last_became_peered":"2024-08-17T02:22:05.131822+0000","last_unstale":"2024-08-17T02:22:05.135434+0000","last_undegraded":"2024-08-17T02:22:05.135434+0000","last_fullsized":"2024-08-17T02:22:05.135434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139793+0000","last_change":"2024-08-17T02:22:05.136258+0000","last_active":"2024-08-17T02:22:05.139793+0000","last_peered":"2024-08-17T02:22:05.139793+0000","last_clean":"2024-08-17T02:22:05.139793+0000","last_became_active":"2024-08-17T02:22:05.135573+0000","last_became_peered":"2024-08-17T02:22:05.135573+0000","last_unstale":"2024-08-17T02:22:05.139793+0000","last_undegraded":"2024-08-17T02:22:05.139793+0000","last_fullsized":"2024-08-17T02:22:05.139793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:36:55.030030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142962+0000","last_change":"2024-08-17T02:22:05.139131+0000","last_active":"2024-08-17T02:22:05.142962+0000","last_peered":"2024-08-17T02:22:05.142962+0000","last_clean":"2024-08-17T02:22:05.142962+0000","last_became_active":"2024-08-17T02:22:05.138484+0000","last_became_peered":"2024-08-17T02:22:05.138484+0000","last_unstale":"2024-08-17T02:22:05.142962+0000","last_undegraded":"2024-08-17T02:22:05.142962+0000","last_fullsized":"2024-08-17T02:22:05.142962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:49:31.875303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134588+0000","last_change":"2024-08-17T02:22:05.128466+0000","last_active":"2024-08-17T02:22:05.134588+0000","last_peered":"2024-08-17T02:22:05.134588+0000","last_clean":"2024-08-17T02:22:05.134588+0000","last_became_active":"2024-08-17T02:22:05.128215+0000","last_became_peered":"2024-08-17T02:22:05.128215+0000","last_unstale":"2024-08-17T02:22:05.134588+0000","last_undegraded":"2024-08-17T02:22:05.134588+0000","last_fullsized":"2024-08-17T02:22:05.134588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:49.930124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133891+0000","last_change":"2024-08-17T02:22:05.131735+0000","last_active":"2024-08-17T02:22:05.133891+0000","last_peered":"2024-08-17T02:22:05.133891+0000","last_clean":"2024-08-17T02:22:05.133891+0000","last_became_active":"2024-08-17T02:22:05.131343+0000","last_became_peered":"2024-08-17T02:22:05.131343+0000","last_unstale":"2024-08-17T02:22:05.133891+0000","last_undegraded":"2024-08-17T02:22:05.133891+0000","last_fullsized":"2024-08-17T02:22:05.133891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:41:42.553817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.128947+0000","last_change":"2024-08-17T02:22:05.124427+0000","last_active":"2024-08-17T02:22:05.128947+0000","last_peered":"2024-08-17T02:22:05.128947+0000","last_clean":"2024-08-17T02:22:05.128947+0000","last_became_active":"2024-08-17T02:22:05.124149+0000","last_became_peered":"2024-08-17T02:22:05.124149+0000","last_unstale":"2024-08-17T02:22:05.128947+0000","last_undegraded":"2024-08-17T02:22:05.128947+0000","last_fullsized":"2024-08-17T02:22:05.128947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:57.001696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.140231+0000","last_change":"2024-08-17T02:22:04.140654+0000","last_active":"2024-08-17T02:22:05.140231+0000","last_peered":"2024-08-17T02:22:05.140231+0000","last_clean":"2024-08-17T02:22:05.140231+0000","last_became_active":"2024-08-17T02:22:04.140316+0000","last_became_peered":"2024-08-17T02:22:04.140316+0000","last_unstale":"2024-08-17T02:22:05.140231+0000","last_undegraded":"2024-08-17T02:22:05.140231+0000","last_fullsized":"2024-08-17T02:22:05.140231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136973+0000","last_change":"2024-08-17T02:22:05.130524+0000","last_active":"2024-08-17T02:22:05.136973+0000","last_peered":"2024-08-17T02:22:05.136973+0000","last_clean":"2024-08-17T02:22:05.136973+0000","last_became_active":"2024-08-17T02:22:05.130188+0000","last_became_peered":"2024-08-17T02:22:05.130188+0000","last_unstale":"2024-08-17T02:22:05.136973+0000","last_undegraded":"2024-08-17T02:22:05.136973+0000","last_fullsized":"2024-08-17T02:22:05.136973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:11:14.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134335+0000","last_change":"2024-08-17T02:22:04.123306+0000","last_active":"2024-08-17T02:22:05.134335+0000","last_peered":"2024-08-17T02:22:05.134335+0000","last_clean":"2024-08-17T02:22:05.134335+0000","last_became_active":"2024-08-17T02:22:04.122968+0000","last_became_peered":"2024-08-17T02:22:04.122968+0000","last_unstale":"2024-08-17T02:22:05.134335+0000","last_undegraded":"2024-08-17T02:22:05.134335+0000","last_fullsized":"2024-08-17T02:22:05.134335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:26:59.050552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.124555+0000","last_change":"2024-08-17T02:22:04.124101+0000","last_active":"2024-08-17T02:22:05.124555+0000","last_peered":"2024-08-17T02:22:05.124555+0000","last_clean":"2024-08-17T02:22:05.124555+0000","last_became_active":"2024-08-17T02:22:04.123756+0000","last_became_peered":"2024-08-17T02:22:04.123756+0000","last_unstale":"2024-08-17T02:22:05.124555+0000","last_undegraded":"2024-08-17T02:22:05.124555+0000","last_fullsized":"2024-08-17T02:22:05.124555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135564+0000","last_change":"2024-08-17T02:22:05.133677+0000","last_active":"2024-08-17T02:22:05.135564+0000","last_peered":"2024-08-17T02:22:05.135564+0000","last_clean":"2024-08-17T02:22:05.135564+0000","last_became_active":"2024-08-17T02:22:05.132419+0000","last_became_peered":"2024-08-17T02:22:05.132419+0000","last_unstale":"2024-08-17T02:22:05.135564+0000","last_undegraded":"2024-08-17T02:22:05.135564+0000","last_fullsized":"2024-08-17T02:22:05.135564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:00:22.156028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.115878+0000","last_change":"2024-08-17T02:22:04.129312+0000","last_active":"2024-08-17T02:22:05.115878+0000","last_peered":"2024-08-17T02:22:05.115878+0000","last_clean":"2024-08-17T02:22:05.115878+0000","last_became_active":"2024-08-17T02:22:04.129003+0000","last_became_peered":"2024-08-17T02:22:04.129003+0000","last_unstale":"2024-08-17T02:22:05.115878+0000","last_undegraded":"2024-08-17T02:22:05.115878+0000","last_fullsized":"2024-08-17T02:22:05.115878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139117+0000","last_change":"2024-08-17T02:22:05.133370+0000","last_active":"2024-08-17T02:22:05.139117+0000","last_peered":"2024-08-17T02:22:05.139117+0000","last_clean":"2024-08-17T02:22:05.139117+0000","last_became_active":"2024-08-17T02:22:05.133041+0000","last_became_peered":"2024-08-17T02:22:05.133041+0000","last_unstale":"2024-08-17T02:22:05.139117+0000","last_undegraded":"2024-08-17T02:22:05.139117+0000","last_fullsized":"2024-08-17T02:22:05.139117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120010+0000","last_change":"2024-08-17T02:22:04.134772+0000","last_active":"2024-08-17T02:22:05.120010+0000","last_peered":"2024-08-17T02:22:05.120010+0000","last_clean":"2024-08-17T02:22:05.120010+0000","last_became_active":"2024-08-17T02:22:04.134509+0000","last_became_peered":"2024-08-17T02:22:04.134509+0000","last_unstale":"2024-08-17T02:22:05.120010+0000","last_undegraded":"2024-08-17T02:22:05.120010+0000","last_fullsized":"2024-08-17T02:22:05.120010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:01.702706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142478+0000","last_change":"2024-08-17T02:22:05.139006+0000","last_active":"2024-08-17T02:22:05.142478+0000","last_peered":"2024-08-17T02:22:05.142478+0000","last_clean":"2024-08-17T02:22:05.142478+0000","last_became_active":"2024-08-17T02:22:05.138609+0000","last_became_peered":"2024-08-17T02:22:05.138609+0000","last_unstale":"2024-08-17T02:22:05.142478+0000","last_undegraded":"2024-08-17T02:22:05.142478+0000","last_fullsized":"2024-08-17T02:22:05.142478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.116705+0000","last_change":"2024-08-17T02:22:04.129251+0000","last_active":"2024-08-17T02:22:05.116705+0000","last_peered":"2024-08-17T02:22:05.116705+0000","last_clean":"2024-08-17T02:22:05.116705+0000","last_became_active":"2024-08-17T02:22:04.126603+0000","last_became_peered":"2024-08-17T02:22:04.126603+0000","last_unstale":"2024-08-17T02:22:05.116705+0000","last_undegraded":"2024-08-17T02:22:05.116705+0000","last_fullsized":"2024-08-17T02:22:05.116705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139326+0000","last_change":"2024-08-17T02:22:05.131334+0000","last_active":"2024-08-17T02:22:05.139326+0000","last_peered":"2024-08-17T02:22:05.139326+0000","last_clean":"2024-08-17T02:22:05.139326+0000","last_became_active":"2024-08-17T02:22:05.130325+0000","last_became_peered":"2024-08-17T02:22:05.130325+0000","last_unstale":"2024-08-17T02:22:05.139326+0000","last_undegraded":"2024-08-17T02:22:05.139326+0000","last_fullsized":"2024-08-17T02:22:05.139326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:47:26.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142458+0000","last_change":"2024-08-17T02:22:04.135036+0000","last_active":"2024-08-17T02:22:05.142458+0000","last_peered":"2024-08-17T02:22:05.142458+0000","last_clean":"2024-08-17T02:22:05.142458+0000","last_became_active":"2024-08-17T02:22:04.134786+0000","last_became_peered":"2024-08-17T02:22:04.134786+0000","last_unstale":"2024-08-17T02:22:05.142458+0000","last_undegraded":"2024-08-17T02:22:05.142458+0000","last_fullsized":"2024-08-17T02:22:05.142458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.997912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146826+0000","last_change":"2024-08-17T02:22:05.141338+0000","last_active":"2024-08-17T02:22:05.146826+0000","last_peered":"2024-08-17T02:22:05.146826+0000","last_clean":"2024-08-17T02:22:05.146826+0000","last_became_active":"2024-08-17T02:22:05.139395+0000","last_became_peered":"2024-08-17T02:22:05.139395+0000","last_unstale":"2024-08-17T02:22:05.146826+0000","last_undegraded":"2024-08-17T02:22:05.146826+0000","last_fullsized":"2024-08-17T02:22:05.146826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:07.057717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146729+0000","last_change":"2024-08-17T02:22:04.127992+0000","last_active":"2024-08-17T02:22:05.146729+0000","last_peered":"2024-08-17T02:22:05.146729+0000","last_clean":"2024-08-17T02:22:05.146729+0000","last_became_active":"2024-08-17T02:22:04.127656+0000","last_became_peered":"2024-08-17T02:22:04.127656+0000","last_unstale":"2024-08-17T02:22:05.146729+0000","last_undegraded":"2024-08-17T02:22:05.146729+0000","last_fullsized":"2024-08-17T02:22:05.146729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:06.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142876+0000","last_change":"2024-08-17T02:22:05.136992+0000","last_active":"2024-08-17T02:22:05.142876+0000","last_peered":"2024-08-17T02:22:05.142876+0000","last_clean":"2024-08-17T02:22:05.142876+0000","last_became_active":"2024-08-17T02:22:05.136611+0000","last_became_peered":"2024-08-17T02:22:05.136611+0000","last_unstale":"2024-08-17T02:22:05.142876+0000","last_undegraded":"2024-08-17T02:22:05.142876+0000","last_fullsized":"2024-08-17T02:22:05.142876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:27:57.075296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147451+0000","last_change":"2024-08-17T02:22:04.136005+0000","last_active":"2024-08-17T02:22:05.147451+0000","last_peered":"2024-08-17T02:22:05.147451+0000","last_clean":"2024-08-17T02:22:05.147451+0000","last_became_active":"2024-08-17T02:22:04.135718+0000","last_became_peered":"2024-08-17T02:22:04.135718+0000","last_unstale":"2024-08-17T02:22:05.147451+0000","last_undegraded":"2024-08-17T02:22:05.147451+0000","last_fullsized":"2024-08-17T02:22:05.147451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:11.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144007+0000","last_change":"2024-08-17T02:22:05.137445+0000","last_active":"2024-08-17T02:22:05.144007+0000","last_peered":"2024-08-17T02:22:05.144007+0000","last_clean":"2024-08-17T02:22:05.144007+0000","last_became_active":"2024-08-17T02:22:05.137047+0000","last_became_peered":"2024-08-17T02:22:05.137047+0000","last_unstale":"2024-08-17T02:22:05.144007+0000","last_undegraded":"2024-08-17T02:22:05.144007+0000","last_fullsized":"2024-08-17T02:22:05.144007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:49.930124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.117051+0000","last_change":"2024-08-17T02:22:04.128882+0000","last_active":"2024-08-17T02:22:05.117051+0000","last_peered":"2024-08-17T02:22:05.117051+0000","last_clean":"2024-08-17T02:22:05.117051+0000","last_became_active":"2024-08-17T02:22:04.124521+0000","last_became_peered":"2024-08-17T02:22:04.124521+0000","last_unstale":"2024-08-17T02:22:05.117051+0000","last_undegraded":"2024-08-17T02:22:05.117051+0000","last_fullsized":"2024-08-17T02:22:05.117051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146565+0000","last_change":"2024-08-17T02:22:05.140259+0000","last_active":"2024-08-17T02:22:05.146565+0000","last_peered":"2024-08-17T02:22:05.146565+0000","last_clean":"2024-08-17T02:22:05.146565+0000","last_became_active":"2024-08-17T02:22:05.139966+0000","last_became_peered":"2024-08-17T02:22:05.139966+0000","last_unstale":"2024-08-17T02:22:05.146565+0000","last_undegraded":"2024-08-17T02:22:05.146565+0000","last_fullsized":"2024-08-17T02:22:05.146565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:01:27.557725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120796+0000","last_change":"2024-08-17T02:22:04.122812+0000","last_active":"2024-08-17T02:22:05.120796+0000","last_peered":"2024-08-17T02:22:05.120796+0000","last_clean":"2024-08-17T02:22:05.120796+0000","last_became_active":"2024-08-17T02:22:04.122506+0000","last_became_peered":"2024-08-17T02:22:04.122506+0000","last_unstale":"2024-08-17T02:22:05.120796+0000","last_undegraded":"2024-08-17T02:22:05.120796+0000","last_fullsized":"2024-08-17T02:22:05.120796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:56.324601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133838+0000","last_change":"2024-08-17T02:22:05.131385+0000","last_active":"2024-08-17T02:22:05.133838+0000","last_peered":"2024-08-17T02:22:05.133838+0000","last_clean":"2024-08-17T02:22:05.133838+0000","last_became_active":"2024-08-17T02:22:05.130986+0000","last_became_peered":"2024-08-17T02:22:05.130986+0000","last_unstale":"2024-08-17T02:22:05.133838+0000","last_undegraded":"2024-08-17T02:22:05.133838+0000","last_fullsized":"2024-08-17T02:22:05.133838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.132407+0000","last_change":"2024-08-17T02:22:04.125802+0000","last_active":"2024-08-17T02:22:05.132407+0000","last_peered":"2024-08-17T02:22:05.132407+0000","last_clean":"2024-08-17T02:22:05.132407+0000","last_became_active":"2024-08-17T02:22:04.125462+0000","last_became_peered":"2024-08-17T02:22:04.125462+0000","last_unstale":"2024-08-17T02:22:05.132407+0000","last_undegraded":"2024-08-17T02:22:05.132407+0000","last_fullsized":"2024-08-17T02:22:05.132407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143008+0000","last_change":"2024-08-17T02:22:05.137082+0000","last_active":"2024-08-17T02:22:05.143008+0000","last_peered":"2024-08-17T02:22:05.143008+0000","last_clean":"2024-08-17T02:22:05.143008+0000","last_became_active":"2024-08-17T02:22:05.136773+0000","last_became_peered":"2024-08-17T02:22:05.136773+0000","last_unstale":"2024-08-17T02:22:05.143008+0000","last_undegraded":"2024-08-17T02:22:05.143008+0000","last_fullsized":"2024-08-17T02:22:05.143008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142302+0000","last_change":"2024-08-17T02:22:04.136447+0000","last_active":"2024-08-17T02:22:05.142302+0000","last_peered":"2024-08-17T02:22:05.142302+0000","last_clean":"2024-08-17T02:22:05.142302+0000","last_became_active":"2024-08-17T02:22:04.136105+0000","last_became_peered":"2024-08-17T02:22:04.136105+0000","last_unstale":"2024-08-17T02:22:05.142302+0000","last_undegraded":"2024-08-17T02:22:05.142302+0000","last_fullsized":"2024-08-17T02:22:05.142302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147858+0000","last_change":"2024-08-17T02:22:05.145138+0000","last_active":"2024-08-17T02:22:05.147858+0000","last_peered":"2024-08-17T02:22:05.147858+0000","last_clean":"2024-08-17T02:22:05.147858+0000","last_became_active":"2024-08-17T02:22:05.144754+0000","last_became_peered":"2024-08-17T02:22:05.144754+0000","last_unstale":"2024-08-17T02:22:05.147858+0000","last_undegraded":"2024-08-17T02:22:05.147858+0000","last_fullsized":"2024-08-17T02:22:05.147858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:36:55.030030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133637+0000","last_change":"2024-08-17T02:22:04.128289+0000","last_active":"2024-08-17T02:22:05.133637+0000","last_peered":"2024-08-17T02:22:05.133637+0000","last_clean":"2024-08-17T02:22:05.133637+0000","last_became_active":"2024-08-17T02:22:04.127984+0000","last_became_peered":"2024-08-17T02:22:04.127984+0000","last_unstale":"2024-08-17T02:22:05.133637+0000","last_undegraded":"2024-08-17T02:22:05.133637+0000","last_fullsized":"2024-08-17T02:22:05.133637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147852+0000","last_change":"2024-08-17T02:22:05.145108+0000","last_active":"2024-08-17T02:22:05.147852+0000","last_peered":"2024-08-17T02:22:05.147852+0000","last_clean":"2024-08-17T02:22:05.147852+0000","last_became_active":"2024-08-17T02:22:05.144771+0000","last_became_peered":"2024-08-17T02:22:05.144771+0000","last_unstale":"2024-08-17T02:22:05.147852+0000","last_undegraded":"2024-08-17T02:22:05.147852+0000","last_fullsized":"2024-08-17T02:22:05.147852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:34:43.102929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139054+0000","last_change":"2024-08-17T02:22:04.125374+0000","last_active":"2024-08-17T02:22:05.139054+0000","last_peered":"2024-08-17T02:22:05.139054+0000","last_clean":"2024-08-17T02:22:05.139054+0000","last_became_active":"2024-08-17T02:22:04.125059+0000","last_became_peered":"2024-08-17T02:22:04.125059+0000","last_unstale":"2024-08-17T02:22:05.139054+0000","last_undegraded":"2024-08-17T02:22:05.139054+0000","last_fullsized":"2024-08-17T02:22:05.139054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:01.702706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142758+0000","last_change":"2024-08-17T02:22:05.139051+0000","last_active":"2024-08-17T02:22:05.142758+0000","last_peered":"2024-08-17T02:22:05.142758+0000","last_clean":"2024-08-17T02:22:05.142758+0000","last_became_active":"2024-08-17T02:22:05.135262+0000","last_became_peered":"2024-08-17T02:22:05.135262+0000","last_unstale":"2024-08-17T02:22:05.142758+0000","last_undegraded":"2024-08-17T02:22:05.142758+0000","last_fullsized":"2024-08-17T02:22:05.142758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:42:24.349708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.118431+0000","last_change":"2024-08-17T02:22:04.137875+0000","last_active":"2024-08-17T02:22:05.118431+0000","last_peered":"2024-08-17T02:22:05.118431+0000","last_clean":"2024-08-17T02:22:05.118431+0000","last_became_active":"2024-08-17T02:22:04.137590+0000","last_became_peered":"2024-08-17T02:22:04.137590+0000","last_unstale":"2024-08-17T02:22:05.118431+0000","last_undegraded":"2024-08-17T02:22:05.118431+0000","last_fullsized":"2024-08-17T02:22:05.118431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146797+0000","last_change":"2024-08-17T02:22:05.142646+0000","last_active":"2024-08-17T02:22:05.146797+0000","last_peered":"2024-08-17T02:22:05.146797+0000","last_clean":"2024-08-17T02:22:05.146797+0000","last_became_active":"2024-08-17T02:22:05.141915+0000","last_became_peered":"2024-08-17T02:22:05.141915+0000","last_unstale":"2024-08-17T02:22:05.146797+0000","last_undegraded":"2024-08-17T02:22:05.146797+0000","last_fullsized":"2024-08-17T02:22:05.146797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:30:37.267266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120860+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:05.120860+0000","last_peered":"2024-08-17T02:22:05.120860+0000","last_clean":"2024-08-17T02:22:05.120860+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:05.120860+0000","last_undegraded":"2024-08-17T02:22:05.120860+0000","last_fullsized":"2024-08-17T02:22:05.120860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.116535+0000","last_change":"2024-08-17T02:22:04.129059+0000","last_active":"2024-08-17T02:22:05.116535+0000","last_peered":"2024-08-17T02:22:05.116535+0000","last_clean":"2024-08-17T02:22:05.116535+0000","last_became_active":"2024-08-17T02:22:04.126173+0000","last_became_peered":"2024-08-17T02:22:04.126173+0000","last_unstale":"2024-08-17T02:22:05.116535+0000","last_undegraded":"2024-08-17T02:22:05.116535+0000","last_fullsized":"2024-08-17T02:22:05.116535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.145732+0000","last_change":"2024-08-17T02:22:05.142911+0000","last_active":"2024-08-17T02:22:05.145732+0000","last_peered":"2024-08-17T02:22:05.145732+0000","last_clean":"2024-08-17T02:22:05.145732+0000","last_became_active":"2024-08-17T02:22:05.142665+0000","last_became_peered":"2024-08-17T02:22:05.142665+0000","last_unstale":"2024-08-17T02:22:05.145732+0000","last_undegraded":"2024-08-17T02:22:05.145732+0000","last_fullsized":"2024-08-17T02:22:05.145732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:41:42.553817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120935+0000","last_change":"2024-08-17T02:22:04.125465+0000","last_active":"2024-08-17T02:22:05.120935+0000","last_peered":"2024-08-17T02:22:05.120935+0000","last_clean":"2024-08-17T02:22:05.120935+0000","last_became_active":"2024-08-17T02:22:04.125205+0000","last_became_peered":"2024-08-17T02:22:04.125205+0000","last_unstale":"2024-08-17T02:22:05.120935+0000","last_undegraded":"2024-08-17T02:22:05.120935+0000","last_fullsized":"2024-08-17T02:22:05.120935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:59:53.518892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139508+0000","last_change":"2024-08-17T02:22:05.132007+0000","last_active":"2024-08-17T02:22:05.139508+0000","last_peered":"2024-08-17T02:22:05.139508+0000","last_clean":"2024-08-17T02:22:05.139508+0000","last_became_active":"2024-08-17T02:22:05.131083+0000","last_became_peered":"2024-08-17T02:22:05.131083+0000","last_unstale":"2024-08-17T02:22:05.139508+0000","last_undegraded":"2024-08-17T02:22:05.139508+0000","last_fullsized":"2024-08-17T02:22:05.139508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:15.303455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.119774+0000","last_change":"2024-08-17T02:22:04.128675+0000","last_active":"2024-08-17T02:22:05.119774+0000","last_peered":"2024-08-17T02:22:05.119774+0000","last_clean":"2024-08-17T02:22:05.119774+0000","last_became_active":"2024-08-17T02:22:04.128264+0000","last_became_peered":"2024-08-17T02:22:04.128264+0000","last_unstale":"2024-08-17T02:22:05.119774+0000","last_undegraded":"2024-08-17T02:22:05.119774+0000","last_fullsized":"2024-08-17T02:22:05.119774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.125552+0000","last_change":"2024-08-17T02:22:05.123180+0000","last_active":"2024-08-17T02:22:05.125552+0000","last_peered":"2024-08-17T02:22:05.125552+0000","last_clean":"2024-08-17T02:22:05.125552+0000","last_became_active":"2024-08-17T02:22:05.122887+0000","last_became_peered":"2024-08-17T02:22:05.122887+0000","last_unstale":"2024-08-17T02:22:05.125552+0000","last_undegraded":"2024-08-17T02:22:05.125552+0000","last_fullsized":"2024-08-17T02:22:05.125552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.117161+0000","last_change":"2024-08-17T02:22:04.125809+0000","last_active":"2024-08-17T02:22:05.117161+0000","last_peered":"2024-08-17T02:22:05.117161+0000","last_clean":"2024-08-17T02:22:05.117161+0000","last_became_active":"2024-08-17T02:22:04.125484+0000","last_became_peered":"2024-08-17T02:22:04.125484+0000","last_unstale":"2024-08-17T02:22:05.117161+0000","last_undegraded":"2024-08-17T02:22:05.117161+0000","last_fullsized":"2024-08-17T02:22:05.117161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146600+0000","last_change":"2024-08-17T02:22:05.138522+0000","last_active":"2024-08-17T02:22:05.146600+0000","last_peered":"2024-08-17T02:22:05.146600+0000","last_clean":"2024-08-17T02:22:05.146600+0000","last_became_active":"2024-08-17T02:22:05.136133+0000","last_became_peered":"2024-08-17T02:22:05.136133+0000","last_unstale":"2024-08-17T02:22:05.146600+0000","last_undegraded":"2024-08-17T02:22:05.146600+0000","last_fullsized":"2024-08-17T02:22:05.146600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:48.803891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110226+0000","last_change":"2024-08-17T02:22:04.139328+0000","last_active":"2024-08-17T02:22:05.110226+0000","last_peered":"2024-08-17T02:22:05.110226+0000","last_clean":"2024-08-17T02:22:05.110226+0000","last_became_active":"2024-08-17T02:22:04.139078+0000","last_became_peered":"2024-08-17T02:22:04.139078+0000","last_unstale":"2024-08-17T02:22:05.110226+0000","last_undegraded":"2024-08-17T02:22:05.110226+0000","last_fullsized":"2024-08-17T02:22:05.110226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:26:59.050552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141791+0000","last_change":"2024-08-17T02:22:05.135288+0000","last_active":"2024-08-17T02:22:05.141791+0000","last_peered":"2024-08-17T02:22:05.141791+0000","last_clean":"2024-08-17T02:22:05.141791+0000","last_became_active":"2024-08-17T02:22:05.134902+0000","last_became_peered":"2024-08-17T02:22:05.134902+0000","last_unstale":"2024-08-17T02:22:05.141791+0000","last_undegraded":"2024-08-17T02:22:05.141791+0000","last_fullsized":"2024-08-17T02:22:05.141791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:41:22.363365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.127565+0000","last_change":"2024-08-17T02:22:04.136037+0000","last_active":"2024-08-17T02:22:05.127565+0000","last_peered":"2024-08-17T02:22:05.127565+0000","last_clean":"2024-08-17T02:22:05.127565+0000","last_became_active":"2024-08-17T02:22:04.135795+0000","last_became_peered":"2024-08-17T02:22:04.135795+0000","last_unstale":"2024-08-17T02:22:05.127565+0000","last_undegraded":"2024-08-17T02:22:05.127565+0000","last_fullsized":"2024-08-17T02:22:05.127565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.124422+0000","last_change":"2024-08-17T02:22:05.123134+0000","last_active":"2024-08-17T02:22:05.124422+0000","last_peered":"2024-08-17T02:22:05.124422+0000","last_clean":"2024-08-17T02:22:05.124422+0000","last_became_active":"2024-08-17T02:22:05.122849+0000","last_became_peered":"2024-08-17T02:22:05.122849+0000","last_unstale":"2024-08-17T02:22:05.124422+0000","last_undegraded":"2024-08-17T02:22:05.124422+0000","last_fullsized":"2024-08-17T02:22:05.124422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:37.064590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110074+0000","last_change":"2024-08-17T02:22:04.124830+0000","last_active":"2024-08-17T02:22:05.110074+0000","last_peered":"2024-08-17T02:22:05.110074+0000","last_clean":"2024-08-17T02:22:05.110074+0000","last_became_active":"2024-08-17T02:22:04.124412+0000","last_became_peered":"2024-08-17T02:22:04.124412+0000","last_unstale":"2024-08-17T02:22:05.110074+0000","last_undegraded":"2024-08-17T02:22:05.110074+0000","last_fullsized":"2024-08-17T02:22:05.110074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:06.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.145579+0000","last_change":"2024-08-17T02:22:05.140560+0000","last_active":"2024-08-17T02:22:05.145579+0000","last_peered":"2024-08-17T02:22:05.145579+0000","last_clean":"2024-08-17T02:22:05.145579+0000","last_became_active":"2024-08-17T02:22:05.140249+0000","last_became_peered":"2024-08-17T02:22:05.140249+0000","last_unstale":"2024-08-17T02:22:05.145579+0000","last_undegraded":"2024-08-17T02:22:05.145579+0000","last_fullsized":"2024-08-17T02:22:05.145579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:24:46.183039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138031+0000","last_change":"2024-08-17T02:22:04.140288+0000","last_active":"2024-08-17T02:22:05.138031+0000","last_peered":"2024-08-17T02:22:05.138031+0000","last_clean":"2024-08-17T02:22:05.138031+0000","last_became_active":"2024-08-17T02:22:04.139999+0000","last_became_peered":"2024-08-17T02:22:04.139999+0000","last_unstale":"2024-08-17T02:22:05.138031+0000","last_undegraded":"2024-08-17T02:22:05.138031+0000","last_fullsized":"2024-08-17T02:22:05.138031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.997912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136517+0000","last_change":"2024-08-17T02:22:05.129259+0000","last_active":"2024-08-17T02:22:05.136517+0000","last_peered":"2024-08-17T02:22:05.136517+0000","last_clean":"2024-08-17T02:22:05.136517+0000","last_became_active":"2024-08-17T02:22:05.128993+0000","last_became_peered":"2024-08-17T02:22:05.128993+0000","last_unstale":"2024-08-17T02:22:05.136517+0000","last_undegraded":"2024-08-17T02:22:05.136517+0000","last_fullsized":"2024-08-17T02:22:05.136517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120637+0000","last_change":"2024-08-17T02:22:04.136465+0000","last_active":"2024-08-17T02:22:05.120637+0000","last_peered":"2024-08-17T02:22:05.120637+0000","last_clean":"2024-08-17T02:22:05.120637+0000","last_became_active":"2024-08-17T02:22:04.136153+0000","last_became_peered":"2024-08-17T02:22:04.136153+0000","last_unstale":"2024-08-17T02:22:05.120637+0000","last_undegraded":"2024-08-17T02:22:05.120637+0000","last_fullsized":"2024-08-17T02:22:05.120637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:25:44.685521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.130033+0000","last_change":"2024-08-17T02:22:05.125665+0000","last_active":"2024-08-17T02:22:05.130033+0000","last_peered":"2024-08-17T02:22:05.130033+0000","last_clean":"2024-08-17T02:22:05.130033+0000","last_became_active":"2024-08-17T02:22:05.125115+0000","last_became_peered":"2024-08-17T02:22:05.125115+0000","last_unstale":"2024-08-17T02:22:05.130033+0000","last_undegraded":"2024-08-17T02:22:05.130033+0000","last_fullsized":"2024-08-17T02:22:05.130033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:57.328385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110749+0000","last_change":"2024-08-17T02:22:04.125704+0000","last_active":"2024-08-17T02:22:05.110749+0000","last_peered":"2024-08-17T02:22:05.110749+0000","last_clean":"2024-08-17T02:22:05.110749+0000","last_became_active":"2024-08-17T02:22:04.124384+0000","last_became_peered":"2024-08-17T02:22:04.124384+0000","last_unstale":"2024-08-17T02:22:05.110749+0000","last_undegraded":"2024-08-17T02:22:05.110749+0000","last_fullsized":"2024-08-17T02:22:05.110749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:56.324601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144755+0000","last_change":"2024-08-17T02:22:05.140080+0000","last_active":"2024-08-17T02:22:05.144755+0000","last_peered":"2024-08-17T02:22:05.144755+0000","last_clean":"2024-08-17T02:22:05.144755+0000","last_became_active":"2024-08-17T02:22:05.139642+0000","last_became_peered":"2024-08-17T02:22:05.139642+0000","last_unstale":"2024-08-17T02:22:05.144755+0000","last_undegraded":"2024-08-17T02:22:05.144755+0000","last_fullsized":"2024-08-17T02:22:05.144755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110708+0000","last_change":"2024-08-17T02:22:04.128418+0000","last_active":"2024-08-17T02:22:05.110708+0000","last_peered":"2024-08-17T02:22:05.110708+0000","last_clean":"2024-08-17T02:22:05.110708+0000","last_became_active":"2024-08-17T02:22:04.128178+0000","last_became_peered":"2024-08-17T02:22:04.128178+0000","last_unstale":"2024-08-17T02:22:05.110708+0000","last_undegraded":"2024-08-17T02:22:05.110708+0000","last_fullsized":"2024-08-17T02:22:05.110708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135568+0000","last_change":"2024-08-17T02:22:05.132678+0000","last_active":"2024-08-17T02:22:05.135568+0000","last_peered":"2024-08-17T02:22:05.135568+0000","last_clean":"2024-08-17T02:22:05.135568+0000","last_became_active":"2024-08-17T02:22:05.132375+0000","last_became_peered":"2024-08-17T02:22:05.132375+0000","last_unstale":"2024-08-17T02:22:05.135568+0000","last_undegraded":"2024-08-17T02:22:05.135568+0000","last_fullsized":"2024-08-17T02:22:05.135568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:05:58.104311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144474+0000","last_change":"2024-08-17T02:22:04.120559+0000","last_active":"2024-08-17T02:22:05.144474+0000","last_peered":"2024-08-17T02:22:05.144474+0000","last_clean":"2024-08-17T02:22:05.144474+0000","last_became_active":"2024-08-17T02:22:04.120229+0000","last_became_peered":"2024-08-17T02:22:04.120229+0000","last_unstale":"2024-08-17T02:22:05.144474+0000","last_undegraded":"2024-08-17T02:22:05.144474+0000","last_fullsized":"2024-08-17T02:22:05.144474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.464171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133762+0000","last_change":"2024-08-17T02:22:05.129982+0000","last_active":"2024-08-17T02:22:05.133762+0000","last_peered":"2024-08-17T02:22:05.133762+0000","last_clean":"2024-08-17T02:22:05.133762+0000","last_became_active":"2024-08-17T02:22:05.129645+0000","last_became_peered":"2024-08-17T02:22:05.129645+0000","last_unstale":"2024-08-17T02:22:05.133762+0000","last_undegraded":"2024-08-17T02:22:05.133762+0000","last_fullsized":"2024-08-17T02:22:05.133762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141475+0000","last_change":"2024-08-17T02:22:05.130378+0000","last_active":"2024-08-17T02:22:05.141475+0000","last_peered":"2024-08-17T02:22:05.141475+0000","last_clean":"2024-08-17T02:22:05.141475+0000","last_became_active":"2024-08-17T02:22:05.130027+0000","last_became_peered":"2024-08-17T02:22:05.130027+0000","last_unstale":"2024-08-17T02:22:05.141475+0000","last_undegraded":"2024-08-17T02:22:05.141475+0000","last_fullsized":"2024-08-17T02:22:05.141475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:44:40.942386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136607+0000","last_change":"2024-08-17T02:22:04.126856+0000","last_active":"2024-08-17T02:22:05.136607+0000","last_peered":"2024-08-17T02:22:05.136607+0000","last_clean":"2024-08-17T02:22:05.136607+0000","last_became_active":"2024-08-17T02:22:04.126479+0000","last_became_peered":"2024-08-17T02:22:04.126479+0000","last_unstale":"2024-08-17T02:22:05.136607+0000","last_undegraded":"2024-08-17T02:22:05.136607+0000","last_fullsized":"2024-08-17T02:22:05.136607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139018+0000","last_change":"2024-08-17T02:22:05.129249+0000","last_active":"2024-08-17T02:22:05.139018+0000","last_peered":"2024-08-17T02:22:05.139018+0000","last_clean":"2024-08-17T02:22:05.139018+0000","last_became_active":"2024-08-17T02:22:05.127468+0000","last_became_peered":"2024-08-17T02:22:05.127468+0000","last_unstale":"2024-08-17T02:22:05.139018+0000","last_undegraded":"2024-08-17T02:22:05.139018+0000","last_fullsized":"2024-08-17T02:22:05.139018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:48:12.922294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.137386+0000","last_change":"2024-08-17T02:22:04.130183+0000","last_active":"2024-08-17T02:22:05.137386+0000","last_peered":"2024-08-17T02:22:05.137386+0000","last_clean":"2024-08-17T02:22:05.137386+0000","last_became_active":"2024-08-17T02:22:04.129922+0000","last_became_peered":"2024-08-17T02:22:04.129922+0000","last_unstale":"2024-08-17T02:22:05.137386+0000","last_undegraded":"2024-08-17T02:22:05.137386+0000","last_fullsized":"2024-08-17T02:22:05.137386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139349+0000","last_change":"2024-08-17T02:22:05.131559+0000","last_active":"2024-08-17T02:22:05.139349+0000","last_peered":"2024-08-17T02:22:05.139349+0000","last_clean":"2024-08-17T02:22:05.139349+0000","last_became_active":"2024-08-17T02:22:05.131142+0000","last_became_peered":"2024-08-17T02:22:05.131142+0000","last_unstale":"2024-08-17T02:22:05.139349+0000","last_undegraded":"2024-08-17T02:22:05.139349+0000","last_fullsized":"2024-08-17T02:22:05.139349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:21:52.626562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134704+0000","last_change":"2024-08-17T02:22:04.136408+0000","last_active":"2024-08-17T02:22:05.134704+0000","last_peered":"2024-08-17T02:22:05.134704+0000","last_clean":"2024-08-17T02:22:05.134704+0000","last_became_active":"2024-08-17T02:22:04.136140+0000","last_became_peered":"2024-08-17T02:22:04.136140+0000","last_unstale":"2024-08-17T02:22:05.134704+0000","last_undegraded":"2024-08-17T02:22:05.134704+0000","last_fullsized":"2024-08-17T02:22:05.134704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138499+0000","last_change":"2024-08-17T02:22:05.134585+0000","last_active":"2024-08-17T02:22:05.138499+0000","last_peered":"2024-08-17T02:22:05.138499+0000","last_clean":"2024-08-17T02:22:05.138499+0000","last_became_active":"2024-08-17T02:22:05.134230+0000","last_became_peered":"2024-08-17T02:22:05.134230+0000","last_unstale":"2024-08-17T02:22:05.138499+0000","last_undegraded":"2024-08-17T02:22:05.138499+0000","last_fullsized":"2024-08-17T02:22:05.138499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144051+0000","last_change":"2024-08-17T02:22:04.140683+0000","last_active":"2024-08-17T02:22:05.144051+0000","last_peered":"2024-08-17T02:22:05.144051+0000","last_clean":"2024-08-17T02:22:05.144051+0000","last_became_active":"2024-08-17T02:22:04.140386+0000","last_became_peered":"2024-08-17T02:22:04.140386+0000","last_unstale":"2024-08-17T02:22:05.144051+0000","last_undegraded":"2024-08-17T02:22:05.144051+0000","last_fullsized":"2024-08-17T02:22:05.144051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298360,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073480,"statfs":{"total":96636764160,"available":96331243520,"internally_reserved":0,"allocated":823296,"data_stored":669312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":229376,"data_stored":78464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":237568,"data_stored":78500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":78374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":233472,"data_stored":78534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":831488,"data_stored":669368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:12.525 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:12.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.636+0000 7ff965a34700 1 Processor -- start 2024-08-17T02:22:12.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.636+0000 7ff965a34700 1 -- start start 2024-08-17T02:22:12.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff960142c60 0x7ff96015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9600cc800 con 0x7ff960142890 2024-08-17T02:22:12.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96015d830 con 0x7ff96015d3e0 2024-08-17T02:22:12.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9600cbbf0 con 0x7ff960142c60 2024-08-17T02:22:12.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95e7fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff960142c60 0x7ff96015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95e7fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff960142c60 0x7ff96015cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56736/0 (socket says 172.21.15.100:56736) 2024-08-17T02:22:12.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95e7fc700 1 -- 172.21.15.100:0/3994628143 learned_addr learned my addr 172.21.15.100:0/3994628143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:12.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95effd700 1 --1- 172.21.15.100:0/3994628143 >> v1:172.21.15.100:6789/0 conn(0x7ff960142890 0x7ff9600cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46980/0 (socket says 172.21.15.100:46980) 2024-08-17T02:22:12.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95e7fc700 1 -- 172.21.15.100:0/3994628143 >> v1:172.21.15.123:6789/0 conn(0x7ff96015d3e0 legacy=0x7ff9600b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95e7fc700 1 -- 172.21.15.100:0/3994628143 >> v1:172.21.15.100:6789/0 conn(0x7ff960142890 legacy=0x7ff9600cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95dffb700 1 -- 172.21.15.100:0/3994628143 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935973321 0 0) 0x7ff9600cc800 con 0x7ff960142890 2024-08-17T02:22:12.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95e7fc700 1 -- 172.21.15.100:0/3994628143 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9600cbed0 con 0x7ff960142c60 2024-08-17T02:22:12.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95e7fc700 1 --2- 172.21.15.100:0/3994628143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015cea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff9500099e0 tx=0x7ff950009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d531be8617a759 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:12.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95dffb700 1 -- 172.21.15.100:0/3994628143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95000b820 con 0x7ff960142c60 2024-08-17T02:22:12.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95dffb700 1 -- 172.21.15.100:0/3994628143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95000b980 con 0x7ff960142c60 2024-08-17T02:22:12.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff95dffb700 1 -- 172.21.15.100:0/3994628143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95000bd20 con 0x7ff960142c60 2024-08-17T02:22:12.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 -- 172.21.15.100:0/3994628143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 msgr2=0x7ff96015cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 --2- 172.21.15.100:0/3994628143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015cea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff9500099e0 tx=0x7ff950009250 comp rx=0 tx=0).stop 2024-08-17T02:22:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 -- 172.21.15.100:0/3994628143 shutdown_connections 2024-08-17T02:22:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 --2- 172.21.15.100:0/3994628143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015cea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 -- 172.21.15.100:0/3994628143 >> 172.21.15.100:0/3994628143 conn(0x7ff9600a9db0 msgr2=0x7ff9600ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 -- 172.21.15.100:0/3994628143 shutdown_connections 2024-08-17T02:22:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.640+0000 7ff965a34700 1 -- 172.21.15.100:0/3994628143 wait complete. 2024-08-17T02:22:12.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.644+0000 7ff965a34700 1 Processor -- start 2024-08-17T02:22:12.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.644+0000 7ff965a34700 1 -- start start 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff960142890 0x7ff96015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff96015d3e0 0x7ff9601402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff960159160 con 0x7ff960142c60 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96015a100 con 0x7ff960142890 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff960155690 con 0x7ff96015d3e0 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56744/0 (socket says 172.21.15.100:56744) 2024-08-17T02:22:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 -- 172.21.15.100:0/2187085354 learned_addr learned my addr 172.21.15.100:0/2187085354 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95effd700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff960142890 0x7ff96015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95f7fe700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff96015d3e0 0x7ff9601402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 -- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff96015d3e0 msgr2=0x7ff9601402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff96015d3e0 0x7ff9601402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 -- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff960142890 msgr2=0x7ff96015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff960142890 0x7ff96015b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 -- 172.21.15.100:0/2187085354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500094f0 con 0x7ff960142c60 2024-08-17T02:22:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff95e7fc700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015c460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9500099e0 tx=0x7ff950017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff946ffd700 1 -- 172.21.15.100:0/2187085354 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95000bd00 con 0x7ff960142c60 2024-08-17T02:22:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff946ffd700 1 -- 172.21.15.100:0/2187085354 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff950024440 con 0x7ff960142c60 2024-08-17T02:22:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff946ffd700 1 -- 172.21.15.100:0/2187085354 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9500247c0 con 0x7ff960142c60 2024-08-17T02:22:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96015b6d0 con 0x7ff960142c60 2024-08-17T02:22:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff960159d90 con 0x7ff960142c60 2024-08-17T02:22:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.648+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff960159910 con 0x7ff960142c60 2024-08-17T02:22:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.652+0000 7ff946ffd700 1 -- 172.21.15.100:0/2187085354 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff95002d440 con 0x7ff960142c60 2024-08-17T02:22:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.652+0000 7ff946ffd700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff94c060860 0x7ff94c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.652+0000 7ff946ffd700 1 -- 172.21.15.100:0/2187085354 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff95007e600 con 0x7ff960142c60 2024-08-17T02:22:12.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.652+0000 7ff95effd700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff94c060860 0x7ff94c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.652+0000 7ff95effd700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff94c060860 0x7ff94c062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff954007fd0 tx=0x7ff954007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:12.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.660+0000 7ff946ffd700 1 -- 172.21.15.100:0/2187085354 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff960159910 con 0x7ff960142c60 2024-08-17T02:22:12.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.824+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff960159910 con 0x7ff94c060860 2024-08-17T02:22:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.840+0000 7ff946ffd700 1 -- 172.21.15.100:0/2187085354 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264496 (secure 0 0 0) 0x7ff960159910 con 0x7ff94c060860 2024-08-17T02:22:12.849 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:12.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff94c060860 msgr2=0x7ff94c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff94c060860 0x7ff94c062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff954007fd0 tx=0x7ff954007d30 comp rx=0 tx=0).stop 2024-08-17T02:22:12.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 msgr2=0x7ff96015c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015c460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9500099e0 tx=0x7ff950017000 comp rx=0 tx=0).stop 2024-08-17T02:22:12.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 shutdown_connections 2024-08-17T02:22:12.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff94c060860 0x7ff94c062d10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff96015d3e0 0x7ff9601402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960142c60 0x7ff96015c460 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 --2- 172.21.15.100:0/2187085354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff960142890 0x7ff96015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 >> 172.21.15.100:0/2187085354 conn(0x7ff9600a9db0 msgr2=0x7ff96013fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:12.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 shutdown_connections 2024-08-17T02:22:12.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.844+0000 7ff965a34700 1 -- 172.21.15.100:0/2187085354 wait complete. 2024-08-17T02:22:12.855 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:12.864 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-08-17T02:22:11.223385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382032,"kb_used_data":2960,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592688,"statfs":{"total":773094113280,"available":770654912512,"internally_reserved":0,"allocated":3031040,"data_stored":1799828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.121991"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143110+0000","last_change":"2024-08-17T02:22:05.137118+0000","last_active":"2024-08-17T02:22:05.143110+0000","last_peered":"2024-08-17T02:22:05.143110+0000","last_clean":"2024-08-17T02:22:05.143110+0000","last_became_active":"2024-08-17T02:22:05.136784+0000","last_became_peered":"2024-08-17T02:22:05.136784+0000","last_unstale":"2024-08-17T02:22:05.143110+0000","last_undegraded":"2024-08-17T02:22:05.143110+0000","last_fullsized":"2024-08-17T02:22:05.143110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:04:25.260837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133391+0000","last_change":"2024-08-17T02:22:05.129636+0000","last_active":"2024-08-17T02:22:05.133391+0000","last_peered":"2024-08-17T02:22:05.133391+0000","last_clean":"2024-08-17T02:22:05.133391+0000","last_became_active":"2024-08-17T02:22:05.129342+0000","last_became_peered":"2024-08-17T02:22:05.129342+0000","last_unstale":"2024-08-17T02:22:05.133391+0000","last_undegraded":"2024-08-17T02:22:05.133391+0000","last_fullsized":"2024-08-17T02:22:05.133391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.131713+0000","last_change":"2024-08-17T02:22:05.128097+0000","last_active":"2024-08-17T02:22:05.131713+0000","last_peered":"2024-08-17T02:22:05.131713+0000","last_clean":"2024-08-17T02:22:05.131713+0000","last_became_active":"2024-08-17T02:22:05.127786+0000","last_became_peered":"2024-08-17T02:22:05.127786+0000","last_unstale":"2024-08-17T02:22:05.131713+0000","last_undegraded":"2024-08-17T02:22:05.131713+0000","last_fullsized":"2024-08-17T02:22:05.131713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:47.605286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142739+0000","last_change":"2024-08-17T02:22:05.138911+0000","last_active":"2024-08-17T02:22:05.142739+0000","last_peered":"2024-08-17T02:22:05.142739+0000","last_clean":"2024-08-17T02:22:05.142739+0000","last_became_active":"2024-08-17T02:22:05.134886+0000","last_became_peered":"2024-08-17T02:22:05.134886+0000","last_unstale":"2024-08-17T02:22:05.142739+0000","last_undegraded":"2024-08-17T02:22:05.142739+0000","last_fullsized":"2024-08-17T02:22:05.142739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:04:25.260837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143121+0000","last_change":"2024-08-17T02:22:05.137212+0000","last_active":"2024-08-17T02:22:05.143121+0000","last_peered":"2024-08-17T02:22:05.143121+0000","last_clean":"2024-08-17T02:22:05.143121+0000","last_became_active":"2024-08-17T02:22:05.136566+0000","last_became_peered":"2024-08-17T02:22:05.136566+0000","last_unstale":"2024-08-17T02:22:05.143121+0000","last_undegraded":"2024-08-17T02:22:05.143121+0000","last_fullsized":"2024-08-17T02:22:05.143121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:49.230407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141537+0000","last_change":"2024-08-17T02:22:05.130974+0000","last_active":"2024-08-17T02:22:05.141537+0000","last_peered":"2024-08-17T02:22:05.141537+0000","last_clean":"2024-08-17T02:22:05.141537+0000","last_became_active":"2024-08-17T02:22:05.130696+0000","last_became_peered":"2024-08-17T02:22:05.130696+0000","last_unstale":"2024-08-17T02:22:05.141537+0000","last_undegraded":"2024-08-17T02:22:05.141537+0000","last_fullsized":"2024-08-17T02:22:05.141537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:49.230407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143027+0000","last_change":"2024-08-17T02:22:05.137564+0000","last_active":"2024-08-17T02:22:05.143027+0000","last_peered":"2024-08-17T02:22:05.143027+0000","last_clean":"2024-08-17T02:22:05.143027+0000","last_became_active":"2024-08-17T02:22:05.137060+0000","last_became_peered":"2024-08-17T02:22:05.137060+0000","last_unstale":"2024-08-17T02:22:05.143027+0000","last_undegraded":"2024-08-17T02:22:05.143027+0000","last_fullsized":"2024-08-17T02:22:05.143027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:48:26.336166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146835+0000","last_change":"2024-08-17T02:22:05.143886+0000","last_active":"2024-08-17T02:22:05.146835+0000","last_peered":"2024-08-17T02:22:05.146835+0000","last_clean":"2024-08-17T02:22:05.146835+0000","last_became_active":"2024-08-17T02:22:05.143537+0000","last_became_peered":"2024-08-17T02:22:05.143537+0000","last_unstale":"2024-08-17T02:22:05.146835+0000","last_undegraded":"2024-08-17T02:22:05.146835+0000","last_fullsized":"2024-08-17T02:22:05.146835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:58:32.777645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146777+0000","last_change":"2024-08-17T02:22:05.142931+0000","last_active":"2024-08-17T02:22:05.146777+0000","last_peered":"2024-08-17T02:22:05.146777+0000","last_clean":"2024-08-17T02:22:05.146777+0000","last_became_active":"2024-08-17T02:22:05.142076+0000","last_became_peered":"2024-08-17T02:22:05.142076+0000","last_unstale":"2024-08-17T02:22:05.146777+0000","last_undegraded":"2024-08-17T02:22:05.146777+0000","last_fullsized":"2024-08-17T02:22:05.146777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:54:13.258567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133864+0000","last_change":"2024-08-17T02:22:05.131490+0000","last_active":"2024-08-17T02:22:05.133864+0000","last_peered":"2024-08-17T02:22:05.133864+0000","last_clean":"2024-08-17T02:22:05.133864+0000","last_became_active":"2024-08-17T02:22:05.131184+0000","last_became_peered":"2024-08-17T02:22:05.131184+0000","last_unstale":"2024-08-17T02:22:05.133864+0000","last_undegraded":"2024-08-17T02:22:05.133864+0000","last_fullsized":"2024-08-17T02:22:05.133864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134444+0000","last_change":"2024-08-17T02:22:05.131423+0000","last_active":"2024-08-17T02:22:05.134444+0000","last_peered":"2024-08-17T02:22:05.134444+0000","last_clean":"2024-08-17T02:22:05.134444+0000","last_became_active":"2024-08-17T02:22:05.131081+0000","last_became_peered":"2024-08-17T02:22:05.131081+0000","last_unstale":"2024-08-17T02:22:05.134444+0000","last_undegraded":"2024-08-17T02:22:05.134444+0000","last_fullsized":"2024-08-17T02:22:05.134444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133327+0000","last_change":"2024-08-17T02:22:05.129487+0000","last_active":"2024-08-17T02:22:05.133327+0000","last_peered":"2024-08-17T02:22:05.133327+0000","last_clean":"2024-08-17T02:22:05.133327+0000","last_became_active":"2024-08-17T02:22:05.129222+0000","last_became_peered":"2024-08-17T02:22:05.129222+0000","last_unstale":"2024-08-17T02:22:05.133327+0000","last_undegraded":"2024-08-17T02:22:05.133327+0000","last_fullsized":"2024-08-17T02:22:05.133327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:02.706489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138694+0000","last_change":"2024-08-17T02:22:05.134693+0000","last_active":"2024-08-17T02:22:05.138694+0000","last_peered":"2024-08-17T02:22:05.138694+0000","last_clean":"2024-08-17T02:22:05.138694+0000","last_became_active":"2024-08-17T02:22:05.134300+0000","last_became_peered":"2024-08-17T02:22:05.134300+0000","last_unstale":"2024-08-17T02:22:05.138694+0000","last_undegraded":"2024-08-17T02:22:05.138694+0000","last_fullsized":"2024-08-17T02:22:05.138694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:37.064590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.131330+0000","last_change":"2024-08-17T02:22:05.128086+0000","last_active":"2024-08-17T02:22:05.131330+0000","last_peered":"2024-08-17T02:22:05.131330+0000","last_clean":"2024-08-17T02:22:05.131330+0000","last_became_active":"2024-08-17T02:22:05.127750+0000","last_became_peered":"2024-08-17T02:22:05.127750+0000","last_unstale":"2024-08-17T02:22:05.131330+0000","last_undegraded":"2024-08-17T02:22:05.131330+0000","last_fullsized":"2024-08-17T02:22:05.131330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:01:27.557725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.123798+0000","last_change":"2024-08-17T02:22:05.123028+0000","last_active":"2024-08-17T02:22:05.123798+0000","last_peered":"2024-08-17T02:22:05.123798+0000","last_clean":"2024-08-17T02:22:05.123798+0000","last_became_active":"2024-08-17T02:22:05.122711+0000","last_became_peered":"2024-08-17T02:22:05.122711+0000","last_unstale":"2024-08-17T02:22:05.123798+0000","last_undegraded":"2024-08-17T02:22:05.123798+0000","last_fullsized":"2024-08-17T02:22:05.123798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141452+0000","last_change":"2024-08-17T02:22:05.130380+0000","last_active":"2024-08-17T02:22:05.141452+0000","last_peered":"2024-08-17T02:22:05.141452+0000","last_clean":"2024-08-17T02:22:05.141452+0000","last_became_active":"2024-08-17T02:22:05.130077+0000","last_became_peered":"2024-08-17T02:22:05.130077+0000","last_unstale":"2024-08-17T02:22:05.141452+0000","last_undegraded":"2024-08-17T02:22:05.141452+0000","last_fullsized":"2024-08-17T02:22:05.141452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:27:57.075296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142733+0000","last_change":"2024-08-17T02:22:05.139113+0000","last_active":"2024-08-17T02:22:05.142733+0000","last_peered":"2024-08-17T02:22:05.142733+0000","last_clean":"2024-08-17T02:22:05.142733+0000","last_became_active":"2024-08-17T02:22:05.138752+0000","last_became_peered":"2024-08-17T02:22:05.138752+0000","last_unstale":"2024-08-17T02:22:05.142733+0000","last_undegraded":"2024-08-17T02:22:05.142733+0000","last_fullsized":"2024-08-17T02:22:05.142733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:21:52.626562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.129913+0000","last_change":"2024-08-17T02:22:05.125930+0000","last_active":"2024-08-17T02:22:05.129913+0000","last_peered":"2024-08-17T02:22:05.129913+0000","last_clean":"2024-08-17T02:22:05.129913+0000","last_became_active":"2024-08-17T02:22:05.125640+0000","last_became_peered":"2024-08-17T02:22:05.125640+0000","last_unstale":"2024-08-17T02:22:05.129913+0000","last_undegraded":"2024-08-17T02:22:05.129913+0000","last_fullsized":"2024-08-17T02:22:05.129913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:59:54.522675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142959+0000","last_change":"2024-08-17T02:22:05.137818+0000","last_active":"2024-08-17T02:22:05.142959+0000","last_peered":"2024-08-17T02:22:05.142959+0000","last_clean":"2024-08-17T02:22:05.142959+0000","last_became_active":"2024-08-17T02:22:05.137531+0000","last_became_peered":"2024-08-17T02:22:05.137531+0000","last_unstale":"2024-08-17T02:22:05.142959+0000","last_undegraded":"2024-08-17T02:22:05.142959+0000","last_fullsized":"2024-08-17T02:22:05.142959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:44:40.942386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.137413+0000","last_change":"2024-08-17T02:22:05.135163+0000","last_active":"2024-08-17T02:22:05.137413+0000","last_peered":"2024-08-17T02:22:05.137413+0000","last_clean":"2024-08-17T02:22:05.137413+0000","last_became_active":"2024-08-17T02:22:05.134647+0000","last_became_peered":"2024-08-17T02:22:05.134647+0000","last_unstale":"2024-08-17T02:22:05.137413+0000","last_undegraded":"2024-08-17T02:22:05.137413+0000","last_fullsized":"2024-08-17T02:22:05.137413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:21.690535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144205+0000","last_change":"2024-08-17T02:22:05.137440+0000","last_active":"2024-08-17T02:22:05.144205+0000","last_peered":"2024-08-17T02:22:05.144205+0000","last_clean":"2024-08-17T02:22:05.144205+0000","last_became_active":"2024-08-17T02:22:05.137081+0000","last_became_peered":"2024-08-17T02:22:05.137081+0000","last_unstale":"2024-08-17T02:22:05.144205+0000","last_undegraded":"2024-08-17T02:22:05.144205+0000","last_fullsized":"2024-08-17T02:22:05.144205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:34:43.102929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142978+0000","last_change":"2024-08-17T02:22:05.139220+0000","last_active":"2024-08-17T02:22:05.142978+0000","last_peered":"2024-08-17T02:22:05.142978+0000","last_clean":"2024-08-17T02:22:05.142978+0000","last_became_active":"2024-08-17T02:22:05.138838+0000","last_became_peered":"2024-08-17T02:22:05.138838+0000","last_unstale":"2024-08-17T02:22:05.142978+0000","last_undegraded":"2024-08-17T02:22:05.142978+0000","last_fullsized":"2024-08-17T02:22:05.142978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:47:26.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135504+0000","last_change":"2024-08-17T02:22:05.132463+0000","last_active":"2024-08-17T02:22:05.135504+0000","last_peered":"2024-08-17T02:22:05.135504+0000","last_clean":"2024-08-17T02:22:05.135504+0000","last_became_active":"2024-08-17T02:22:05.131741+0000","last_became_peered":"2024-08-17T02:22:05.131741+0000","last_unstale":"2024-08-17T02:22:05.135504+0000","last_undegraded":"2024-08-17T02:22:05.135504+0000","last_fullsized":"2024-08-17T02:22:05.135504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142777+0000","last_change":"2024-08-17T02:22:05.140510+0000","last_active":"2024-08-17T02:22:05.142777+0000","last_peered":"2024-08-17T02:22:05.142777+0000","last_clean":"2024-08-17T02:22:05.142777+0000","last_became_active":"2024-08-17T02:22:05.139145+0000","last_became_peered":"2024-08-17T02:22:05.139145+0000","last_unstale":"2024-08-17T02:22:05.142777+0000","last_undegraded":"2024-08-17T02:22:05.142777+0000","last_fullsized":"2024-08-17T02:22:05.142777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:58:53.092915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135524+0000","last_change":"2024-08-17T02:22:05.133736+0000","last_active":"2024-08-17T02:22:05.135524+0000","last_peered":"2024-08-17T02:22:05.135524+0000","last_clean":"2024-08-17T02:22:05.135524+0000","last_became_active":"2024-08-17T02:22:05.132607+0000","last_became_peered":"2024-08-17T02:22:05.132607+0000","last_unstale":"2024-08-17T02:22:05.135524+0000","last_undegraded":"2024-08-17T02:22:05.135524+0000","last_fullsized":"2024-08-17T02:22:05.135524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141845+0000","last_change":"2024-08-17T02:22:05.137747+0000","last_active":"2024-08-17T02:22:05.141845+0000","last_peered":"2024-08-17T02:22:05.141845+0000","last_clean":"2024-08-17T02:22:05.141845+0000","last_became_active":"2024-08-17T02:22:05.137464+0000","last_became_peered":"2024-08-17T02:22:05.137464+0000","last_unstale":"2024-08-17T02:22:05.141845+0000","last_undegraded":"2024-08-17T02:22:05.141845+0000","last_fullsized":"2024-08-17T02:22:05.141845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138959+0000","last_change":"2024-08-17T02:22:05.133235+0000","last_active":"2024-08-17T02:22:05.138959+0000","last_peered":"2024-08-17T02:22:05.138959+0000","last_clean":"2024-08-17T02:22:05.138959+0000","last_became_active":"2024-08-17T02:22:05.132962+0000","last_became_peered":"2024-08-17T02:22:05.132962+0000","last_unstale":"2024-08-17T02:22:05.138959+0000","last_undegraded":"2024-08-17T02:22:05.138959+0000","last_fullsized":"2024-08-17T02:22:05.138959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135434+0000","last_change":"2024-08-17T02:22:05.132315+0000","last_active":"2024-08-17T02:22:05.135434+0000","last_peered":"2024-08-17T02:22:05.135434+0000","last_clean":"2024-08-17T02:22:05.135434+0000","last_became_active":"2024-08-17T02:22:05.131822+0000","last_became_peered":"2024-08-17T02:22:05.131822+0000","last_unstale":"2024-08-17T02:22:05.135434+0000","last_undegraded":"2024-08-17T02:22:05.135434+0000","last_fullsized":"2024-08-17T02:22:05.135434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139793+0000","last_change":"2024-08-17T02:22:05.136258+0000","last_active":"2024-08-17T02:22:05.139793+0000","last_peered":"2024-08-17T02:22:05.139793+0000","last_clean":"2024-08-17T02:22:05.139793+0000","last_became_active":"2024-08-17T02:22:05.135573+0000","last_became_peered":"2024-08-17T02:22:05.135573+0000","last_unstale":"2024-08-17T02:22:05.139793+0000","last_undegraded":"2024-08-17T02:22:05.139793+0000","last_fullsized":"2024-08-17T02:22:05.139793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:36:55.030030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142962+0000","last_change":"2024-08-17T02:22:05.139131+0000","last_active":"2024-08-17T02:22:05.142962+0000","last_peered":"2024-08-17T02:22:05.142962+0000","last_clean":"2024-08-17T02:22:05.142962+0000","last_became_active":"2024-08-17T02:22:05.138484+0000","last_became_peered":"2024-08-17T02:22:05.138484+0000","last_unstale":"2024-08-17T02:22:05.142962+0000","last_undegraded":"2024-08-17T02:22:05.142962+0000","last_fullsized":"2024-08-17T02:22:05.142962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:49:31.875303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134588+0000","last_change":"2024-08-17T02:22:05.128466+0000","last_active":"2024-08-17T02:22:05.134588+0000","last_peered":"2024-08-17T02:22:05.134588+0000","last_clean":"2024-08-17T02:22:05.134588+0000","last_became_active":"2024-08-17T02:22:05.128215+0000","last_became_peered":"2024-08-17T02:22:05.128215+0000","last_unstale":"2024-08-17T02:22:05.134588+0000","last_undegraded":"2024-08-17T02:22:05.134588+0000","last_fullsized":"2024-08-17T02:22:05.134588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:49.930124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133891+0000","last_change":"2024-08-17T02:22:05.131735+0000","last_active":"2024-08-17T02:22:05.133891+0000","last_peered":"2024-08-17T02:22:05.133891+0000","last_clean":"2024-08-17T02:22:05.133891+0000","last_became_active":"2024-08-17T02:22:05.131343+0000","last_became_peered":"2024-08-17T02:22:05.131343+0000","last_unstale":"2024-08-17T02:22:05.133891+0000","last_undegraded":"2024-08-17T02:22:05.133891+0000","last_fullsized":"2024-08-17T02:22:05.133891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:41:42.553817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.128947+0000","last_change":"2024-08-17T02:22:05.124427+0000","last_active":"2024-08-17T02:22:05.128947+0000","last_peered":"2024-08-17T02:22:05.128947+0000","last_clean":"2024-08-17T02:22:05.128947+0000","last_became_active":"2024-08-17T02:22:05.124149+0000","last_became_peered":"2024-08-17T02:22:05.124149+0000","last_unstale":"2024-08-17T02:22:05.128947+0000","last_undegraded":"2024-08-17T02:22:05.128947+0000","last_fullsized":"2024-08-17T02:22:05.128947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:57.001696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.140231+0000","last_change":"2024-08-17T02:22:04.140654+0000","last_active":"2024-08-17T02:22:05.140231+0000","last_peered":"2024-08-17T02:22:05.140231+0000","last_clean":"2024-08-17T02:22:05.140231+0000","last_became_active":"2024-08-17T02:22:04.140316+0000","last_became_peered":"2024-08-17T02:22:04.140316+0000","last_unstale":"2024-08-17T02:22:05.140231+0000","last_undegraded":"2024-08-17T02:22:05.140231+0000","last_fullsized":"2024-08-17T02:22:05.140231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136973+0000","last_change":"2024-08-17T02:22:05.130524+0000","last_active":"2024-08-17T02:22:05.136973+0000","last_peered":"2024-08-17T02:22:05.136973+0000","last_clean":"2024-08-17T02:22:05.136973+0000","last_became_active":"2024-08-17T02:22:05.130188+0000","last_became_peered":"2024-08-17T02:22:05.130188+0000","last_unstale":"2024-08-17T02:22:05.136973+0000","last_undegraded":"2024-08-17T02:22:05.136973+0000","last_fullsized":"2024-08-17T02:22:05.136973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:11:14.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134335+0000","last_change":"2024-08-17T02:22:04.123306+0000","last_active":"2024-08-17T02:22:05.134335+0000","last_peered":"2024-08-17T02:22:05.134335+0000","last_clean":"2024-08-17T02:22:05.134335+0000","last_became_active":"2024-08-17T02:22:04.122968+0000","last_became_peered":"2024-08-17T02:22:04.122968+0000","last_unstale":"2024-08-17T02:22:05.134335+0000","last_undegraded":"2024-08-17T02:22:05.134335+0000","last_fullsized":"2024-08-17T02:22:05.134335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:26:59.050552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.124555+0000","last_change":"2024-08-17T02:22:04.124101+0000","last_active":"2024-08-17T02:22:05.124555+0000","last_peered":"2024-08-17T02:22:05.124555+0000","last_clean":"2024-08-17T02:22:05.124555+0000","last_became_active":"2024-08-17T02:22:04.123756+0000","last_became_peered":"2024-08-17T02:22:04.123756+0000","last_unstale":"2024-08-17T02:22:05.124555+0000","last_undegraded":"2024-08-17T02:22:05.124555+0000","last_fullsized":"2024-08-17T02:22:05.124555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135564+0000","last_change":"2024-08-17T02:22:05.133677+0000","last_active":"2024-08-17T02:22:05.135564+0000","last_peered":"2024-08-17T02:22:05.135564+0000","last_clean":"2024-08-17T02:22:05.135564+0000","last_became_active":"2024-08-17T02:22:05.132419+0000","last_became_peered":"2024-08-17T02:22:05.132419+0000","last_unstale":"2024-08-17T02:22:05.135564+0000","last_undegraded":"2024-08-17T02:22:05.135564+0000","last_fullsized":"2024-08-17T02:22:05.135564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:00:22.156028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.115878+0000","last_change":"2024-08-17T02:22:04.129312+0000","last_active":"2024-08-17T02:22:05.115878+0000","last_peered":"2024-08-17T02:22:05.115878+0000","last_clean":"2024-08-17T02:22:05.115878+0000","last_became_active":"2024-08-17T02:22:04.129003+0000","last_became_peered":"2024-08-17T02:22:04.129003+0000","last_unstale":"2024-08-17T02:22:05.115878+0000","last_undegraded":"2024-08-17T02:22:05.115878+0000","last_fullsized":"2024-08-17T02:22:05.115878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139117+0000","last_change":"2024-08-17T02:22:05.133370+0000","last_active":"2024-08-17T02:22:05.139117+0000","last_peered":"2024-08-17T02:22:05.139117+0000","last_clean":"2024-08-17T02:22:05.139117+0000","last_became_active":"2024-08-17T02:22:05.133041+0000","last_became_peered":"2024-08-17T02:22:05.133041+0000","last_unstale":"2024-08-17T02:22:05.139117+0000","last_undegraded":"2024-08-17T02:22:05.139117+0000","last_fullsized":"2024-08-17T02:22:05.139117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120010+0000","last_change":"2024-08-17T02:22:04.134772+0000","last_active":"2024-08-17T02:22:05.120010+0000","last_peered":"2024-08-17T02:22:05.120010+0000","last_clean":"2024-08-17T02:22:05.120010+0000","last_became_active":"2024-08-17T02:22:04.134509+0000","last_became_peered":"2024-08-17T02:22:04.134509+0000","last_unstale":"2024-08-17T02:22:05.120010+0000","last_undegraded":"2024-08-17T02:22:05.120010+0000","last_fullsized":"2024-08-17T02:22:05.120010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:01.702706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142478+0000","last_change":"2024-08-17T02:22:05.139006+0000","last_active":"2024-08-17T02:22:05.142478+0000","last_peered":"2024-08-17T02:22:05.142478+0000","last_clean":"2024-08-17T02:22:05.142478+0000","last_became_active":"2024-08-17T02:22:05.138609+0000","last_became_peered":"2024-08-17T02:22:05.138609+0000","last_unstale":"2024-08-17T02:22:05.142478+0000","last_undegraded":"2024-08-17T02:22:05.142478+0000","last_fullsized":"2024-08-17T02:22:05.142478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.116705+0000","last_change":"2024-08-17T02:22:04.129251+0000","last_active":"2024-08-17T02:22:05.116705+0000","last_peered":"2024-08-17T02:22:05.116705+0000","last_clean":"2024-08-17T02:22:05.116705+0000","last_became_active":"2024-08-17T02:22:04.126603+0000","last_became_peered":"2024-08-17T02:22:04.126603+0000","last_unstale":"2024-08-17T02:22:05.116705+0000","last_undegraded":"2024-08-17T02:22:05.116705+0000","last_fullsized":"2024-08-17T02:22:05.116705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139326+0000","last_change":"2024-08-17T02:22:05.131334+0000","last_active":"2024-08-17T02:22:05.139326+0000","last_peered":"2024-08-17T02:22:05.139326+0000","last_clean":"2024-08-17T02:22:05.139326+0000","last_became_active":"2024-08-17T02:22:05.130325+0000","last_became_peered":"2024-08-17T02:22:05.130325+0000","last_unstale":"2024-08-17T02:22:05.139326+0000","last_undegraded":"2024-08-17T02:22:05.139326+0000","last_fullsized":"2024-08-17T02:22:05.139326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:47:26.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142458+0000","last_change":"2024-08-17T02:22:04.135036+0000","last_active":"2024-08-17T02:22:05.142458+0000","last_peered":"2024-08-17T02:22:05.142458+0000","last_clean":"2024-08-17T02:22:05.142458+0000","last_became_active":"2024-08-17T02:22:04.134786+0000","last_became_peered":"2024-08-17T02:22:04.134786+0000","last_unstale":"2024-08-17T02:22:05.142458+0000","last_undegraded":"2024-08-17T02:22:05.142458+0000","last_fullsized":"2024-08-17T02:22:05.142458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.997912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146826+0000","last_change":"2024-08-17T02:22:05.141338+0000","last_active":"2024-08-17T02:22:05.146826+0000","last_peered":"2024-08-17T02:22:05.146826+0000","last_clean":"2024-08-17T02:22:05.146826+0000","last_became_active":"2024-08-17T02:22:05.139395+0000","last_became_peered":"2024-08-17T02:22:05.139395+0000","last_unstale":"2024-08-17T02:22:05.146826+0000","last_undegraded":"2024-08-17T02:22:05.146826+0000","last_fullsized":"2024-08-17T02:22:05.146826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:07.057717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146729+0000","last_change":"2024-08-17T02:22:04.127992+0000","last_active":"2024-08-17T02:22:05.146729+0000","last_peered":"2024-08-17T02:22:05.146729+0000","last_clean":"2024-08-17T02:22:05.146729+0000","last_became_active":"2024-08-17T02:22:04.127656+0000","last_became_peered":"2024-08-17T02:22:04.127656+0000","last_unstale":"2024-08-17T02:22:05.146729+0000","last_undegraded":"2024-08-17T02:22:05.146729+0000","last_fullsized":"2024-08-17T02:22:05.146729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:06.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142876+0000","last_change":"2024-08-17T02:22:05.136992+0000","last_active":"2024-08-17T02:22:05.142876+0000","last_peered":"2024-08-17T02:22:05.142876+0000","last_clean":"2024-08-17T02:22:05.142876+0000","last_became_active":"2024-08-17T02:22:05.136611+0000","last_became_peered":"2024-08-17T02:22:05.136611+0000","last_unstale":"2024-08-17T02:22:05.142876+0000","last_undegraded":"2024-08-17T02:22:05.142876+0000","last_fullsized":"2024-08-17T02:22:05.142876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:27:57.075296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147451+0000","last_change":"2024-08-17T02:22:04.136005+0000","last_active":"2024-08-17T02:22:05.147451+0000","last_peered":"2024-08-17T02:22:05.147451+0000","last_clean":"2024-08-17T02:22:05.147451+0000","last_became_active":"2024-08-17T02:22:04.135718+0000","last_became_peered":"2024-08-17T02:22:04.135718+0000","last_unstale":"2024-08-17T02:22:05.147451+0000","last_undegraded":"2024-08-17T02:22:05.147451+0000","last_fullsized":"2024-08-17T02:22:05.147451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:11.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144007+0000","last_change":"2024-08-17T02:22:05.137445+0000","last_active":"2024-08-17T02:22:05.144007+0000","last_peered":"2024-08-17T02:22:05.144007+0000","last_clean":"2024-08-17T02:22:05.144007+0000","last_became_active":"2024-08-17T02:22:05.137047+0000","last_became_peered":"2024-08-17T02:22:05.137047+0000","last_unstale":"2024-08-17T02:22:05.144007+0000","last_undegraded":"2024-08-17T02:22:05.144007+0000","last_fullsized":"2024-08-17T02:22:05.144007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:49.930124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.117051+0000","last_change":"2024-08-17T02:22:04.128882+0000","last_active":"2024-08-17T02:22:05.117051+0000","last_peered":"2024-08-17T02:22:05.117051+0000","last_clean":"2024-08-17T02:22:05.117051+0000","last_became_active":"2024-08-17T02:22:04.124521+0000","last_became_peered":"2024-08-17T02:22:04.124521+0000","last_unstale":"2024-08-17T02:22:05.117051+0000","last_undegraded":"2024-08-17T02:22:05.117051+0000","last_fullsized":"2024-08-17T02:22:05.117051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146565+0000","last_change":"2024-08-17T02:22:05.140259+0000","last_active":"2024-08-17T02:22:05.146565+0000","last_peered":"2024-08-17T02:22:05.146565+0000","last_clean":"2024-08-17T02:22:05.146565+0000","last_became_active":"2024-08-17T02:22:05.139966+0000","last_became_peered":"2024-08-17T02:22:05.139966+0000","last_unstale":"2024-08-17T02:22:05.146565+0000","last_undegraded":"2024-08-17T02:22:05.146565+0000","last_fullsized":"2024-08-17T02:22:05.146565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:01:27.557725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120796+0000","last_change":"2024-08-17T02:22:04.122812+0000","last_active":"2024-08-17T02:22:05.120796+0000","last_peered":"2024-08-17T02:22:05.120796+0000","last_clean":"2024-08-17T02:22:05.120796+0000","last_became_active":"2024-08-17T02:22:04.122506+0000","last_became_peered":"2024-08-17T02:22:04.122506+0000","last_unstale":"2024-08-17T02:22:05.120796+0000","last_undegraded":"2024-08-17T02:22:05.120796+0000","last_fullsized":"2024-08-17T02:22:05.120796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:56.324601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133838+0000","last_change":"2024-08-17T02:22:05.131385+0000","last_active":"2024-08-17T02:22:05.133838+0000","last_peered":"2024-08-17T02:22:05.133838+0000","last_clean":"2024-08-17T02:22:05.133838+0000","last_became_active":"2024-08-17T02:22:05.130986+0000","last_became_peered":"2024-08-17T02:22:05.130986+0000","last_unstale":"2024-08-17T02:22:05.133838+0000","last_undegraded":"2024-08-17T02:22:05.133838+0000","last_fullsized":"2024-08-17T02:22:05.133838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.132407+0000","last_change":"2024-08-17T02:22:04.125802+0000","last_active":"2024-08-17T02:22:05.132407+0000","last_peered":"2024-08-17T02:22:05.132407+0000","last_clean":"2024-08-17T02:22:05.132407+0000","last_became_active":"2024-08-17T02:22:04.125462+0000","last_became_peered":"2024-08-17T02:22:04.125462+0000","last_unstale":"2024-08-17T02:22:05.132407+0000","last_undegraded":"2024-08-17T02:22:05.132407+0000","last_fullsized":"2024-08-17T02:22:05.132407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143008+0000","last_change":"2024-08-17T02:22:05.137082+0000","last_active":"2024-08-17T02:22:05.143008+0000","last_peered":"2024-08-17T02:22:05.143008+0000","last_clean":"2024-08-17T02:22:05.143008+0000","last_became_active":"2024-08-17T02:22:05.136773+0000","last_became_peered":"2024-08-17T02:22:05.136773+0000","last_unstale":"2024-08-17T02:22:05.143008+0000","last_undegraded":"2024-08-17T02:22:05.143008+0000","last_fullsized":"2024-08-17T02:22:05.143008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142302+0000","last_change":"2024-08-17T02:22:04.136447+0000","last_active":"2024-08-17T02:22:05.142302+0000","last_peered":"2024-08-17T02:22:05.142302+0000","last_clean":"2024-08-17T02:22:05.142302+0000","last_became_active":"2024-08-17T02:22:04.136105+0000","last_became_peered":"2024-08-17T02:22:04.136105+0000","last_unstale":"2024-08-17T02:22:05.142302+0000","last_undegraded":"2024-08-17T02:22:05.142302+0000","last_fullsized":"2024-08-17T02:22:05.142302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147858+0000","last_change":"2024-08-17T02:22:05.145138+0000","last_active":"2024-08-17T02:22:05.147858+0000","last_peered":"2024-08-17T02:22:05.147858+0000","last_clean":"2024-08-17T02:22:05.147858+0000","last_became_active":"2024-08-17T02:22:05.144754+0000","last_became_peered":"2024-08-17T02:22:05.144754+0000","last_unstale":"2024-08-17T02:22:05.147858+0000","last_undegraded":"2024-08-17T02:22:05.147858+0000","last_fullsized":"2024-08-17T02:22:05.147858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:36:55.030030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133637+0000","last_change":"2024-08-17T02:22:04.128289+0000","last_active":"2024-08-17T02:22:05.133637+0000","last_peered":"2024-08-17T02:22:05.133637+0000","last_clean":"2024-08-17T02:22:05.133637+0000","last_became_active":"2024-08-17T02:22:04.127984+0000","last_became_peered":"2024-08-17T02:22:04.127984+0000","last_unstale":"2024-08-17T02:22:05.133637+0000","last_undegraded":"2024-08-17T02:22:05.133637+0000","last_fullsized":"2024-08-17T02:22:05.133637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147852+0000","last_change":"2024-08-17T02:22:05.145108+0000","last_active":"2024-08-17T02:22:05.147852+0000","last_peered":"2024-08-17T02:22:05.147852+0000","last_clean":"2024-08-17T02:22:05.147852+0000","last_became_active":"2024-08-17T02:22:05.144771+0000","last_became_peered":"2024-08-17T02:22:05.144771+0000","last_unstale":"2024-08-17T02:22:05.147852+0000","last_undegraded":"2024-08-17T02:22:05.147852+0000","last_fullsized":"2024-08-17T02:22:05.147852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:34:43.102929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139054+0000","last_change":"2024-08-17T02:22:04.125374+0000","last_active":"2024-08-17T02:22:05.139054+0000","last_peered":"2024-08-17T02:22:05.139054+0000","last_clean":"2024-08-17T02:22:05.139054+0000","last_became_active":"2024-08-17T02:22:04.125059+0000","last_became_peered":"2024-08-17T02:22:04.125059+0000","last_unstale":"2024-08-17T02:22:05.139054+0000","last_undegraded":"2024-08-17T02:22:05.139054+0000","last_fullsized":"2024-08-17T02:22:05.139054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:01.702706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142758+0000","last_change":"2024-08-17T02:22:05.139051+0000","last_active":"2024-08-17T02:22:05.142758+0000","last_peered":"2024-08-17T02:22:05.142758+0000","last_clean":"2024-08-17T02:22:05.142758+0000","last_became_active":"2024-08-17T02:22:05.135262+0000","last_became_peered":"2024-08-17T02:22:05.135262+0000","last_unstale":"2024-08-17T02:22:05.142758+0000","last_undegraded":"2024-08-17T02:22:05.142758+0000","last_fullsized":"2024-08-17T02:22:05.142758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:42:24.349708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.118431+0000","last_change":"2024-08-17T02:22:04.137875+0000","last_active":"2024-08-17T02:22:05.118431+0000","last_peered":"2024-08-17T02:22:05.118431+0000","last_clean":"2024-08-17T02:22:05.118431+0000","last_became_active":"2024-08-17T02:22:04.137590+0000","last_became_peered":"2024-08-17T02:22:04.137590+0000","last_unstale":"2024-08-17T02:22:05.118431+0000","last_undegraded":"2024-08-17T02:22:05.118431+0000","last_fullsized":"2024-08-17T02:22:05.118431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146797+0000","last_change":"2024-08-17T02:22:05.142646+0000","last_active":"2024-08-17T02:22:05.146797+0000","last_peered":"2024-08-17T02:22:05.146797+0000","last_clean":"2024-08-17T02:22:05.146797+0000","last_became_active":"2024-08-17T02:22:05.141915+0000","last_became_peered":"2024-08-17T02:22:05.141915+0000","last_unstale":"2024-08-17T02:22:05.146797+0000","last_undegraded":"2024-08-17T02:22:05.146797+0000","last_fullsized":"2024-08-17T02:22:05.146797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:30:37.267266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120860+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:05.120860+0000","last_peered":"2024-08-17T02:22:05.120860+0000","last_clean":"2024-08-17T02:22:05.120860+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:05.120860+0000","last_undegraded":"2024-08-17T02:22:05.120860+0000","last_fullsized":"2024-08-17T02:22:05.120860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.116535+0000","last_change":"2024-08-17T02:22:04.129059+0000","last_active":"2024-08-17T02:22:05.116535+0000","last_peered":"2024-08-17T02:22:05.116535+0000","last_clean":"2024-08-17T02:22:05.116535+0000","last_became_active":"2024-08-17T02:22:04.126173+0000","last_became_peered":"2024-08-17T02:22:04.126173+0000","last_unstale":"2024-08-17T02:22:05.116535+0000","last_undegraded":"2024-08-17T02:22:05.116535+0000","last_fullsized":"2024-08-17T02:22:05.116535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.145732+0000","last_change":"2024-08-17T02:22:05.142911+0000","last_active":"2024-08-17T02:22:05.145732+0000","last_peered":"2024-08-17T02:22:05.145732+0000","last_clean":"2024-08-17T02:22:05.145732+0000","last_became_active":"2024-08-17T02:22:05.142665+0000","last_became_peered":"2024-08-17T02:22:05.142665+0000","last_unstale":"2024-08-17T02:22:05.145732+0000","last_undegraded":"2024-08-17T02:22:05.145732+0000","last_fullsized":"2024-08-17T02:22:05.145732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:41:42.553817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120935+0000","last_change":"2024-08-17T02:22:04.125465+0000","last_active":"2024-08-17T02:22:05.120935+0000","last_peered":"2024-08-17T02:22:05.120935+0000","last_clean":"2024-08-17T02:22:05.120935+0000","last_became_active":"2024-08-17T02:22:04.125205+0000","last_became_peered":"2024-08-17T02:22:04.125205+0000","last_unstale":"2024-08-17T02:22:05.120935+0000","last_undegraded":"2024-08-17T02:22:05.120935+0000","last_fullsized":"2024-08-17T02:22:05.120935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:59:53.518892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139508+0000","last_change":"2024-08-17T02:22:05.132007+0000","last_active":"2024-08-17T02:22:05.139508+0000","last_peered":"2024-08-17T02:22:05.139508+0000","last_clean":"2024-08-17T02:22:05.139508+0000","last_became_active":"2024-08-17T02:22:05.131083+0000","last_became_peered":"2024-08-17T02:22:05.131083+0000","last_unstale":"2024-08-17T02:22:05.139508+0000","last_undegraded":"2024-08-17T02:22:05.139508+0000","last_fullsized":"2024-08-17T02:22:05.139508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:15.303455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.119774+0000","last_change":"2024-08-17T02:22:04.128675+0000","last_active":"2024-08-17T02:22:05.119774+0000","last_peered":"2024-08-17T02:22:05.119774+0000","last_clean":"2024-08-17T02:22:05.119774+0000","last_became_active":"2024-08-17T02:22:04.128264+0000","last_became_peered":"2024-08-17T02:22:04.128264+0000","last_unstale":"2024-08-17T02:22:05.119774+0000","last_undegraded":"2024-08-17T02:22:05.119774+0000","last_fullsized":"2024-08-17T02:22:05.119774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.125552+0000","last_change":"2024-08-17T02:22:05.123180+0000","last_active":"2024-08-17T02:22:05.125552+0000","last_peered":"2024-08-17T02:22:05.125552+0000","last_clean":"2024-08-17T02:22:05.125552+0000","last_became_active":"2024-08-17T02:22:05.122887+0000","last_became_peered":"2024-08-17T02:22:05.122887+0000","last_unstale":"2024-08-17T02:22:05.125552+0000","last_undegraded":"2024-08-17T02:22:05.125552+0000","last_fullsized":"2024-08-17T02:22:05.125552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.117161+0000","last_change":"2024-08-17T02:22:04.125809+0000","last_active":"2024-08-17T02:22:05.117161+0000","last_peered":"2024-08-17T02:22:05.117161+0000","last_clean":"2024-08-17T02:22:05.117161+0000","last_became_active":"2024-08-17T02:22:04.125484+0000","last_became_peered":"2024-08-17T02:22:04.125484+0000","last_unstale":"2024-08-17T02:22:05.117161+0000","last_undegraded":"2024-08-17T02:22:05.117161+0000","last_fullsized":"2024-08-17T02:22:05.117161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146600+0000","last_change":"2024-08-17T02:22:05.138522+0000","last_active":"2024-08-17T02:22:05.146600+0000","last_peered":"2024-08-17T02:22:05.146600+0000","last_clean":"2024-08-17T02:22:05.146600+0000","last_became_active":"2024-08-17T02:22:05.136133+0000","last_became_peered":"2024-08-17T02:22:05.136133+0000","last_unstale":"2024-08-17T02:22:05.146600+0000","last_undegraded":"2024-08-17T02:22:05.146600+0000","last_fullsized":"2024-08-17T02:22:05.146600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:48.803891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110226+0000","last_change":"2024-08-17T02:22:04.139328+0000","last_active":"2024-08-17T02:22:05.110226+0000","last_peered":"2024-08-17T02:22:05.110226+0000","last_clean":"2024-08-17T02:22:05.110226+0000","last_became_active":"2024-08-17T02:22:04.139078+0000","last_became_peered":"2024-08-17T02:22:04.139078+0000","last_unstale":"2024-08-17T02:22:05.110226+0000","last_undegraded":"2024-08-17T02:22:05.110226+0000","last_fullsized":"2024-08-17T02:22:05.110226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:26:59.050552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141791+0000","last_change":"2024-08-17T02:22:05.135288+0000","last_active":"2024-08-17T02:22:05.141791+0000","last_peered":"2024-08-17T02:22:05.141791+0000","last_clean":"2024-08-17T02:22:05.141791+0000","last_became_active":"2024-08-17T02:22:05.134902+0000","last_became_peered":"2024-08-17T02:22:05.134902+0000","last_unstale":"2024-08-17T02:22:05.141791+0000","last_undegraded":"2024-08-17T02:22:05.141791+0000","last_fullsized":"2024-08-17T02:22:05.141791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:41:22.363365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.127565+0000","last_change":"2024-08-17T02:22:04.136037+0000","last_active":"2024-08-17T02:22:05.127565+0000","last_peered":"2024-08-17T02:22:05.127565+0000","last_clean":"2024-08-17T02:22:05.127565+0000","last_became_active":"2024-08-17T02:22:04.135795+0000","last_became_peered":"2024-08-17T02:22:04.135795+0000","last_unstale":"2024-08-17T02:22:05.127565+0000","last_undegraded":"2024-08-17T02:22:05.127565+0000","last_fullsized":"2024-08-17T02:22:05.127565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.124422+0000","last_change":"2024-08-17T02:22:05.123134+0000","last_active":"2024-08-17T02:22:05.124422+0000","last_peered":"2024-08-17T02:22:05.124422+0000","last_clean":"2024-08-17T02:22:05.124422+0000","last_became_active":"2024-08-17T02:22:05.122849+0000","last_became_peered":"2024-08-17T02:22:05.122849+0000","last_unstale":"2024-08-17T02:22:05.124422+0000","last_undegraded":"2024-08-17T02:22:05.124422+0000","last_fullsized":"2024-08-17T02:22:05.124422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:37.064590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110074+0000","last_change":"2024-08-17T02:22:04.124830+0000","last_active":"2024-08-17T02:22:05.110074+0000","last_peered":"2024-08-17T02:22:05.110074+0000","last_clean":"2024-08-17T02:22:05.110074+0000","last_became_active":"2024-08-17T02:22:04.124412+0000","last_became_peered":"2024-08-17T02:22:04.124412+0000","last_unstale":"2024-08-17T02:22:05.110074+0000","last_undegraded":"2024-08-17T02:22:05.110074+0000","last_fullsized":"2024-08-17T02:22:05.110074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:06.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.145579+0000","last_change":"2024-08-17T02:22:05.140560+0000","last_active":"2024-08-17T02:22:05.145579+0000","last_peered":"2024-08-17T02:22:05.145579+0000","last_clean":"2024-08-17T02:22:05.145579+0000","last_became_active":"2024-08-17T02:22:05.140249+0000","last_became_peered":"2024-08-17T02:22:05.140249+0000","last_unstale":"2024-08-17T02:22:05.145579+0000","last_undegraded":"2024-08-17T02:22:05.145579+0000","last_fullsized":"2024-08-17T02:22:05.145579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:24:46.183039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138031+0000","last_change":"2024-08-17T02:22:04.140288+0000","last_active":"2024-08-17T02:22:05.138031+0000","last_peered":"2024-08-17T02:22:05.138031+0000","last_clean":"2024-08-17T02:22:05.138031+0000","last_became_active":"2024-08-17T02:22:04.139999+0000","last_became_peered":"2024-08-17T02:22:04.139999+0000","last_unstale":"2024-08-17T02:22:05.138031+0000","last_undegraded":"2024-08-17T02:22:05.138031+0000","last_fullsized":"2024-08-17T02:22:05.138031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.997912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136517+0000","last_change":"2024-08-17T02:22:05.129259+0000","last_active":"2024-08-17T02:22:05.136517+0000","last_peered":"2024-08-17T02:22:05.136517+0000","last_clean":"2024-08-17T02:22:05.136517+0000","last_became_active":"2024-08-17T02:22:05.128993+0000","last_became_peered":"2024-08-17T02:22:05.128993+0000","last_unstale":"2024-08-17T02:22:05.136517+0000","last_undegraded":"2024-08-17T02:22:05.136517+0000","last_fullsized":"2024-08-17T02:22:05.136517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120637+0000","last_change":"2024-08-17T02:22:04.136465+0000","last_active":"2024-08-17T02:22:05.120637+0000","last_peered":"2024-08-17T02:22:05.120637+0000","last_clean":"2024-08-17T02:22:05.120637+0000","last_became_active":"2024-08-17T02:22:04.136153+0000","last_became_peered":"2024-08-17T02:22:04.136153+0000","last_unstale":"2024-08-17T02:22:05.120637+0000","last_undegraded":"2024-08-17T02:22:05.120637+0000","last_fullsized":"2024-08-17T02:22:05.120637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:25:44.685521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.130033+0000","last_change":"2024-08-17T02:22:05.125665+0000","last_active":"2024-08-17T02:22:05.130033+0000","last_peered":"2024-08-17T02:22:05.130033+0000","last_clean":"2024-08-17T02:22:05.130033+0000","last_became_active":"2024-08-17T02:22:05.125115+0000","last_became_peered":"2024-08-17T02:22:05.125115+0000","last_unstale":"2024-08-17T02:22:05.130033+0000","last_undegraded":"2024-08-17T02:22:05.130033+0000","last_fullsized":"2024-08-17T02:22:05.130033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:57.328385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110749+0000","last_change":"2024-08-17T02:22:04.125704+0000","last_active":"2024-08-17T02:22:05.110749+0000","last_peered":"2024-08-17T02:22:05.110749+0000","last_clean":"2024-08-17T02:22:05.110749+0000","last_became_active":"2024-08-17T02:22:04.124384+0000","last_became_peered":"2024-08-17T02:22:04.124384+0000","last_unstale":"2024-08-17T02:22:05.110749+0000","last_undegraded":"2024-08-17T02:22:05.110749+0000","last_fullsized":"2024-08-17T02:22:05.110749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:56.324601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144755+0000","last_change":"2024-08-17T02:22:05.140080+0000","last_active":"2024-08-17T02:22:05.144755+0000","last_peered":"2024-08-17T02:22:05.144755+0000","last_clean":"2024-08-17T02:22:05.144755+0000","last_became_active":"2024-08-17T02:22:05.139642+0000","last_became_peered":"2024-08-17T02:22:05.139642+0000","last_unstale":"2024-08-17T02:22:05.144755+0000","last_undegraded":"2024-08-17T02:22:05.144755+0000","last_fullsized":"2024-08-17T02:22:05.144755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110708+0000","last_change":"2024-08-17T02:22:04.128418+0000","last_active":"2024-08-17T02:22:05.110708+0000","last_peered":"2024-08-17T02:22:05.110708+0000","last_clean":"2024-08-17T02:22:05.110708+0000","last_became_active":"2024-08-17T02:22:04.128178+0000","last_became_peered":"2024-08-17T02:22:04.128178+0000","last_unstale":"2024-08-17T02:22:05.110708+0000","last_undegraded":"2024-08-17T02:22:05.110708+0000","last_fullsized":"2024-08-17T02:22:05.110708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135568+0000","last_change":"2024-08-17T02:22:05.132678+0000","last_active":"2024-08-17T02:22:05.135568+0000","last_peered":"2024-08-17T02:22:05.135568+0000","last_clean":"2024-08-17T02:22:05.135568+0000","last_became_active":"2024-08-17T02:22:05.132375+0000","last_became_peered":"2024-08-17T02:22:05.132375+0000","last_unstale":"2024-08-17T02:22:05.135568+0000","last_undegraded":"2024-08-17T02:22:05.135568+0000","last_fullsized":"2024-08-17T02:22:05.135568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:05:58.104311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144474+0000","last_change":"2024-08-17T02:22:04.120559+0000","last_active":"2024-08-17T02:22:05.144474+0000","last_peered":"2024-08-17T02:22:05.144474+0000","last_clean":"2024-08-17T02:22:05.144474+0000","last_became_active":"2024-08-17T02:22:04.120229+0000","last_became_peered":"2024-08-17T02:22:04.120229+0000","last_unstale":"2024-08-17T02:22:05.144474+0000","last_undegraded":"2024-08-17T02:22:05.144474+0000","last_fullsized":"2024-08-17T02:22:05.144474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.464171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133762+0000","last_change":"2024-08-17T02:22:05.129982+0000","last_active":"2024-08-17T02:22:05.133762+0000","last_peered":"2024-08-17T02:22:05.133762+0000","last_clean":"2024-08-17T02:22:05.133762+0000","last_became_active":"2024-08-17T02:22:05.129645+0000","last_became_peered":"2024-08-17T02:22:05.129645+0000","last_unstale":"2024-08-17T02:22:05.133762+0000","last_undegraded":"2024-08-17T02:22:05.133762+0000","last_fullsized":"2024-08-17T02:22:05.133762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141475+0000","last_change":"2024-08-17T02:22:05.130378+0000","last_active":"2024-08-17T02:22:05.141475+0000","last_peered":"2024-08-17T02:22:05.141475+0000","last_clean":"2024-08-17T02:22:05.141475+0000","last_became_active":"2024-08-17T02:22:05.130027+0000","last_became_peered":"2024-08-17T02:22:05.130027+0000","last_unstale":"2024-08-17T02:22:05.141475+0000","last_undegraded":"2024-08-17T02:22:05.141475+0000","last_fullsized":"2024-08-17T02:22:05.141475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:44:40.942386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136607+0000","last_change":"2024-08-17T02:22:04.126856+0000","last_active":"2024-08-17T02:22:05.136607+0000","last_peered":"2024-08-17T02:22:05.136607+0000","last_clean":"2024-08-17T02:22:05.136607+0000","last_became_active":"2024-08-17T02:22:04.126479+0000","last_became_peered":"2024-08-17T02:22:04.126479+0000","last_unstale":"2024-08-17T02:22:05.136607+0000","last_undegraded":"2024-08-17T02:22:05.136607+0000","last_fullsized":"2024-08-17T02:22:05.136607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139018+0000","last_change":"2024-08-17T02:22:05.129249+0000","last_active":"2024-08-17T02:22:05.139018+0000","last_peered":"2024-08-17T02:22:05.139018+0000","last_clean":"2024-08-17T02:22:05.139018+0000","last_became_active":"2024-08-17T02:22:05.127468+0000","last_became_peered":"2024-08-17T02:22:05.127468+0000","last_unstale":"2024-08-17T02:22:05.139018+0000","last_undegraded":"2024-08-17T02:22:05.139018+0000","last_fullsized":"2024-08-17T02:22:05.139018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:48:12.922294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.137386+0000","last_change":"2024-08-17T02:22:04.130183+0000","last_active":"2024-08-17T02:22:05.137386+0000","last_peered":"2024-08-17T02:22:05.137386+0000","last_clean":"2024-08-17T02:22:05.137386+0000","last_became_active":"2024-08-17T02:22:04.129922+0000","last_became_peered":"2024-08-17T02:22:04.129922+0000","last_unstale":"2024-08-17T02:22:05.137386+0000","last_undegraded":"2024-08-17T02:22:05.137386+0000","last_fullsized":"2024-08-17T02:22:05.137386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139349+0000","last_change":"2024-08-17T02:22:05.131559+0000","last_active":"2024-08-17T02:22:05.139349+0000","last_peered":"2024-08-17T02:22:05.139349+0000","last_clean":"2024-08-17T02:22:05.139349+0000","last_became_active":"2024-08-17T02:22:05.131142+0000","last_became_peered":"2024-08-17T02:22:05.131142+0000","last_unstale":"2024-08-17T02:22:05.139349+0000","last_undegraded":"2024-08-17T02:22:05.139349+0000","last_fullsized":"2024-08-17T02:22:05.139349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:21:52.626562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134704+0000","last_change":"2024-08-17T02:22:04.136408+0000","last_active":"2024-08-17T02:22:05.134704+0000","last_peered":"2024-08-17T02:22:05.134704+0000","last_clean":"2024-08-17T02:22:05.134704+0000","last_became_active":"2024-08-17T02:22:04.136140+0000","last_became_peered":"2024-08-17T02:22:04.136140+0000","last_unstale":"2024-08-17T02:22:05.134704+0000","last_undegraded":"2024-08-17T02:22:05.134704+0000","last_fullsized":"2024-08-17T02:22:05.134704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138499+0000","last_change":"2024-08-17T02:22:05.134585+0000","last_active":"2024-08-17T02:22:05.138499+0000","last_peered":"2024-08-17T02:22:05.138499+0000","last_clean":"2024-08-17T02:22:05.138499+0000","last_became_active":"2024-08-17T02:22:05.134230+0000","last_became_peered":"2024-08-17T02:22:05.134230+0000","last_unstale":"2024-08-17T02:22:05.138499+0000","last_undegraded":"2024-08-17T02:22:05.138499+0000","last_fullsized":"2024-08-17T02:22:05.138499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144051+0000","last_change":"2024-08-17T02:22:04.140683+0000","last_active":"2024-08-17T02:22:05.144051+0000","last_peered":"2024-08-17T02:22:05.144051+0000","last_clean":"2024-08-17T02:22:05.144051+0000","last_became_active":"2024-08-17T02:22:04.140386+0000","last_became_peered":"2024-08-17T02:22:04.140386+0000","last_unstale":"2024-08-17T02:22:05.144051+0000","last_undegraded":"2024-08-17T02:22:05.144051+0000","last_fullsized":"2024-08-17T02:22:05.144051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298360,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073480,"statfs":{"total":96636764160,"available":96331243520,"internally_reserved":0,"allocated":823296,"data_stored":669312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":229376,"data_stored":78464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":237568,"data_stored":78500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":78374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":233472,"data_stored":78534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":831488,"data_stored":669368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:12.872 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-17T02:22:12.872 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-17T02:22:12.872 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-08-17T02:22:12.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-08-17T02:22:12.873 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-17T02:22:12.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.980+0000 7f828a99b700 1 Processor -- start 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.980+0000 7f828a99b700 1 -- start start 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8284142c30 0x7f8284159be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82840b2220 con 0x7f828406ee30 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82840b2500 con 0x7f8284142860 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8284159790 con 0x7f8284142c30 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a19a700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f828406ee30 0x7f828415a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46992/0 (socket says 172.21.15.100:46992) 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a19a700 1 -- 172.21.15.100:0/521142937 learned_addr learned my addr 172.21.15.100:0/521142937 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8289198700 1 --2- 172.21.15.100:0/521142937 >> v2:172.21.15.123:3300/0 conn(0x7f8284142c30 0x7f8284159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8288997700 1 -- 172.21.15.100:0/521142937 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688125063 0 0) 0x7f82840b2220 con 0x7f828406ee30 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8288997700 1 -- 172.21.15.100:0/521142937 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8270003570 con 0x7f828406ee30 2024-08-17T02:22:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8289198700 1 -- 172.21.15.100:0/521142937 >> v1:172.21.15.123:6789/0 conn(0x7f8284142860 legacy=0x7f828406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8288997700 1 -- 172.21.15.100:0/521142937 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4246454500 0 0) 0x7f8270003570 con 0x7f828406ee30 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8289198700 1 -- 172.21.15.100:0/521142937 >> v1:172.21.15.100:6789/0 conn(0x7f828406ee30 legacy=0x7f828415a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8289198700 1 -- 172.21.15.100:0/521142937 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82840b27e0 con 0x7f8284142c30 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8289198700 1 --2- 172.21.15.100:0/521142937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142c30 0x7f8284159be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f82740098e0 tx=0x7f82740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91eca6e147d002e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8288997700 1 -- 172.21.15.100:0/521142937 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c0017f0 con 0x7f8284142c30 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8288997700 1 -- 172.21.15.100:0/521142937 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274011420 con 0x7f8284142c30 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f8288997700 1 -- 172.21.15.100:0/521142937 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740117a0 con 0x7f8284142c30 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 -- 172.21.15.100:0/521142937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142c30 msgr2=0x7f8284159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 --2- 172.21.15.100:0/521142937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142c30 0x7f8284159be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f82740098e0 tx=0x7f82740091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 -- 172.21.15.100:0/521142937 shutdown_connections 2024-08-17T02:22:12.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 --2- 172.21.15.100:0/521142937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142c30 0x7f8284159be0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 -- 172.21.15.100:0/521142937 >> 172.21.15.100:0/521142937 conn(0x7f82840ca2b0 msgr2=0x7f828406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:12.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 -- 172.21.15.100:0/521142937 shutdown_connections 2024-08-17T02:22:12.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.984+0000 7f828a99b700 1 -- 172.21.15.100:0/521142937 wait complete. 2024-08-17T02:22:12.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 Processor -- start 2024-08-17T02:22:12.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 -- start start 2024-08-17T02:22:12.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828406ee30 0x7f828413a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142860 0x7f8284140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8284142c30 0x7f8284140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82840a55f0 con 0x7f828406ee30 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82840c3980 con 0x7f8284142860 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82840c4500 con 0x7f8284142c30 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828406ee30 0x7f828413a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828406ee30 0x7f828413a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56750/0 (socket says 172.21.15.100:56750) 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 -- 172.21.15.100:0/2289100713 learned_addr learned my addr 172.21.15.100:0/2289100713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289198700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142860 0x7f8284140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a19a700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8284142c30 0x7f8284140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 -- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8284142c30 msgr2=0x7f8284140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8284142c30 0x7f8284140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 -- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142860 msgr2=0x7f8284140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142860 0x7f8284140970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 -- 172.21.15.100:0/2289100713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274009420 con 0x7f828406ee30 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289198700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142860 0x7f8284140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8289999700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828406ee30 0x7f828413a970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f828000c5c0 tx=0x7f828000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8279ffb700 1 -- 172.21.15.100:0/2289100713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280012630 con 0x7f828406ee30 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8279ffb700 1 -- 172.21.15.100:0/2289100713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82800127b0 con 0x7f828406ee30 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f8279ffb700 1 -- 172.21.15.100:0/2289100713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280012b70 con 0x7f828406ee30 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82840c3bd0 con 0x7f828406ee30 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.988+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82840c4190 con 0x7f828406ee30 2024-08-17T02:22:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.992+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82840643a0 con 0x7f828406ee30 2024-08-17T02:22:12.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.992+0000 7f8279ffb700 1 -- 172.21.15.100:0/2289100713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f828001b400 con 0x7f828406ee30 2024-08-17T02:22:12.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.992+0000 7f8279ffb700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8268060860 0x7f8268062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:12.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.992+0000 7f8279ffb700 1 -- 172.21.15.100:0/2289100713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f828007c830 con 0x7f828406ee30 2024-08-17T02:22:12.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.992+0000 7f8289198700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8268060860 0x7f8268062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:12.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:12.992+0000 7f8289198700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8268060860 0x7f8268062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8274009fd0 tx=0x7f8274016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.000+0000 7f8279ffb700 1 -- 172.21.15.100:0/2289100713 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f828004cc80 con 0x7f828406ee30 2024-08-17T02:22:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.200+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f82840b11b0 con 0x7f828406ee30 2024-08-17T02:22:13.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.204+0000 7f8279ffb700 1 -- 172.21.15.100:0/2289100713 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154541 (secure 0 0 0) 0x7f828007c070 con 0x7f828406ee30 2024-08-17T02:22:13.209 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:13.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8268060860 msgr2=0x7f8268062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8268060860 0x7f8268062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8274009fd0 tx=0x7f8274016000 comp rx=0 tx=0).stop 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828406ee30 msgr2=0x7f828413a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828406ee30 0x7f828413a970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f828000c5c0 tx=0x7f828000c980 comp rx=0 tx=0).stop 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 shutdown_connections 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8268060860 0x7f8268062d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8284142c30 0x7f8284140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8284142860 0x7f8284140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 --2- 172.21.15.100:0/2289100713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828406ee30 0x7f828413a970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 >> 172.21.15.100:0/2289100713 conn(0x7f82840ca2b0 msgr2=0x7f82840c21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 shutdown_connections 2024-08-17T02:22:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.208+0000 7f828a99b700 1 -- 172.21.15.100:0/2289100713 wait complete. 2024-08-17T02:22:13.224 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":24094},{"type":"v1","addr":"172.21.15.123:6833","nonce":24094}]},"active_addr":"172.21.15.123:6833/24094","active_change":"2024-08-17T02:21:51.202983+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2792473277}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":887213758}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2985653948}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1306285931}]}]} 2024-08-17T02:22:13.227 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-08-17T02:22:13.227 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-08-17T02:22:13.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:13.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1128f95700 1 Processor -- start 2024-08-17T02:22:13.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1128f95700 1 -- start start 2024-08-17T02:22:13.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1128f95700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f11240cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1128f95700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f11240d0010 0x7f11240d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1128f95700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11240cf190 con 0x7f112415cea0 2024-08-17T02:22:13.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1128f95700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11240d09a0 con 0x7f11240d0010 2024-08-17T02:22:13.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1128f95700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11240ce5e0 con 0x7f1124159040 2024-08-17T02:22:13.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1122ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f11240d0010 0x7f11240d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f11237fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f11240cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1123fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f112415cea0 0x7f112415d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:54512/0 (socket says 172.21.15.100:54512) 2024-08-17T02:22:13.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1123fff700 1 -- 172.21.15.100:0/21090709 learned_addr learned my addr 172.21.15.100:0/21090709 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:13.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1122ffd700 1 -- 172.21.15.100:0/21090709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 msgr2=0x7f11240cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:13.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1122ffd700 1 --2- 172.21.15.100:0/21090709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f11240cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1122ffd700 1 -- 172.21.15.100:0/21090709 >> v1:172.21.15.123:6789/0 conn(0x7f112415cea0 legacy=0x7f112415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1122ffd700 1 -- 172.21.15.100:0/21090709 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11240ce860 con 0x7f11240d0010 2024-08-17T02:22:13.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f1122ffd700 1 --2- 172.21.15.100:0/21090709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 0x7f11240d0460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f110c009e60 tx=0x7f110c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bdef72ba6f05b8e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f11227fc700 1 -- 172.21.15.100:0/21090709 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c00e070 con 0x7f11240d0010 2024-08-17T02:22:13.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f11227fc700 1 -- 172.21.15.100:0/21090709 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c00bb60 con 0x7f11240d0010 2024-08-17T02:22:13.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.328+0000 7f11227fc700 1 -- 172.21.15.100:0/21090709 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c012430 con 0x7f11240d0010 2024-08-17T02:22:13.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- 172.21.15.100:0/21090709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 msgr2=0x7f11240d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 --2- 172.21.15.100:0/21090709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 0x7f11240d0460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f110c009e60 tx=0x7f110c009780 comp rx=0 tx=0).stop 2024-08-17T02:22:13.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- 172.21.15.100:0/21090709 shutdown_connections 2024-08-17T02:22:13.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 --2- 172.21.15.100:0/21090709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 0x7f11240d0460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 --2- 172.21.15.100:0/21090709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f11240cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- 172.21.15.100:0/21090709 >> 172.21.15.100:0/21090709 conn(0x7f1124068250 msgr2=0x7f1124068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:13.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- 172.21.15.100:0/21090709 shutdown_connections 2024-08-17T02:22:13.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- 172.21.15.100:0/21090709 wait complete. 2024-08-17T02:22:13.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 Processor -- start 2024-08-17T02:22:13.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- start start 2024-08-17T02:22:13.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 0x7f112416a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f1124152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112415cea0 0x7f112416aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112416a7d0 con 0x7f112415cea0 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11240b1620 con 0x7f11240d0010 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11240a6d90 con 0x7f1124159040 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1123fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112415cea0 0x7f112416aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1122ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f1124152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f11237fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 0x7f112416a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1123fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112415cea0 0x7f112416aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56762/0 (socket says 172.21.15.100:56762) 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f11237fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 0x7f112416a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:45756/0 (socket says 172.21.15.100:45756) 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1122ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f1124152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:51618/0 (socket says 172.21.15.100:51618) 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f11237fe700 1 -- 172.21.15.100:0/2510019247 learned_addr learned my addr 172.21.15.100:0/2510019247 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1123fff700 1 -- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 msgr2=0x7f1124152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1123fff700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f1124152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1123fff700 1 -- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 msgr2=0x7f112416a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1123fff700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 0x7f112416a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1123fff700 1 -- 172.21.15.100:0/2510019247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c009a20 con 0x7f112415cea0 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1123fff700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112415cea0 0x7f112416aa20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1114001eb0 tx=0x7f1114007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f111b7fe700 1 -- 172.21.15.100:0/2510019247 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114005920 con 0x7f112415cea0 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f111b7fe700 1 -- 172.21.15.100:0/2510019247 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1114005a80 con 0x7f112415cea0 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.332+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11240a7070 con 0x7f112415cea0 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.336+0000 7f111b7fe700 1 -- 172.21.15.100:0/2510019247 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114005e40 con 0x7f112415cea0 2024-08-17T02:22:13.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.336+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11240a7630 con 0x7f112415cea0 2024-08-17T02:22:13.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.336+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11240643a0 con 0x7f112415cea0 2024-08-17T02:22:13.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.336+0000 7f111b7fe700 1 -- 172.21.15.100:0/2510019247 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f11140072e0 con 0x7f112415cea0 2024-08-17T02:22:13.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.336+0000 7f111b7fe700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1108060860 0x7f1108062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.336+0000 7f111b7fe700 1 -- 172.21.15.100:0/2510019247 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f111407bd20 con 0x7f112415cea0 2024-08-17T02:22:13.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.336+0000 7f11237fe700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1108060860 0x7f1108062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.340+0000 7f11237fe700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1108060860 0x7f1108062d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f111c009e70 tx=0x7f111c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.344+0000 7f111b7fe700 1 -- 172.21.15.100:0/2510019247 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f111404c170 con 0x7f112415cea0 2024-08-17T02:22:13.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.508+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11240b3480 con 0x7f112415cea0 2024-08-17T02:22:13.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.508+0000 7f111b7fe700 1 -- 172.21.15.100:0/2510019247 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f111404f790 con 0x7f112415cea0 2024-08-17T02:22:13.513 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:13.513 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1108060860 msgr2=0x7f1108062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1108060860 0x7f1108062d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f111c009e70 tx=0x7f111c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112415cea0 msgr2=0x7f112416aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112415cea0 0x7f112416aa20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1114001eb0 tx=0x7f1114007e60 comp rx=0 tx=0).stop 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 shutdown_connections 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1108060860 0x7f1108062d10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112415cea0 0x7f112416aa20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1124159040 0x7f1124152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 --2- 172.21.15.100:0/2510019247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11240d0010 0x7f112416a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 >> 172.21.15.100:0/2510019247 conn(0x7f1124068250 msgr2=0x7f1124157fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 shutdown_connections 2024-08-17T02:22:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.512+0000 7f1128f95700 1 -- 172.21.15.100:0/2510019247 wait complete. 2024-08-17T02:22:13.527 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-08-17T02:22:13.528 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:13.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.628+0000 7fbbf441e700 1 Processor -- start 2024-08-17T02:22:13.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.628+0000 7fbbf441e700 1 -- start start 2024-08-17T02:22:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf441e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec153a40 0x7fbbec0c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf441e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbec0c9e00 con 0x7fbbec0b29f0 2024-08-17T02:22:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf441e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbec0cc890 con 0x7fbbec15a260 2024-08-17T02:22:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf441e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec0ccd20 con 0x7fbbec153a40 2024-08-17T02:22:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf19b9700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbbec0b29f0 0x7fbbec159b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47004/0 (socket says 172.21.15.100:47004) 2024-08-17T02:22:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf19b9700 1 -- 172.21.15.100:0/864796766 learned_addr learned my addr 172.21.15.100:0/864796766 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf21ba700 1 --2- 172.21.15.100:0/864796766 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec153a40 0x7fbbec0c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf11b8700 1 -- 172.21.15.100:0/864796766 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955722156 0 0) 0x7fbbec0c9e00 con 0x7fbbec0b29f0 2024-08-17T02:22:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf11b8700 1 -- 172.21.15.100:0/864796766 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd8003570 con 0x7fbbec0b29f0 2024-08-17T02:22:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf11b8700 1 -- 172.21.15.100:0/864796766 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757079930 0 0) 0x7fbbec0cc890 con 0x7fbbec15a260 2024-08-17T02:22:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf11b8700 1 -- 172.21.15.100:0/864796766 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbec0c9e00 con 0x7fbbec15a260 2024-08-17T02:22:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf21ba700 1 -- 172.21.15.100:0/864796766 >> v1:172.21.15.123:6789/0 conn(0x7fbbec15a260 legacy=0x7fbbec0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf21ba700 1 -- 172.21.15.100:0/864796766 >> v1:172.21.15.100:6789/0 conn(0x7fbbec0b29f0 legacy=0x7fbbec159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf11b8700 1 -- 172.21.15.100:0/864796766 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542096835 0 0) 0x7fbbd8003570 con 0x7fbbec0b29f0 2024-08-17T02:22:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf21ba700 1 -- 172.21.15.100:0/864796766 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec0cbf60 con 0x7fbbec153a40 2024-08-17T02:22:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf21ba700 1 --2- 172.21.15.100:0/864796766 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec153a40 0x7fbbec0c9820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009910 tx=0x7fbbe80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1280868e9e022552 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf11b8700 1 -- 172.21.15.100:0/864796766 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc001470 con 0x7fbbec153a40 2024-08-17T02:22:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf11b8700 1 -- 172.21.15.100:0/864796766 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8011420 con 0x7fbbec153a40 2024-08-17T02:22:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.632+0000 7fbbf11b8700 1 -- 172.21.15.100:0/864796766 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80117c0 con 0x7fbbec153a40 2024-08-17T02:22:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 -- 172.21.15.100:0/864796766 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec153a40 msgr2=0x7fbbec0c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 --2- 172.21.15.100:0/864796766 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec153a40 0x7fbbec0c9820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009910 tx=0x7fbbe80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 -- 172.21.15.100:0/864796766 shutdown_connections 2024-08-17T02:22:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 --2- 172.21.15.100:0/864796766 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec153a40 0x7fbbec0c9820 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 -- 172.21.15.100:0/864796766 >> 172.21.15.100:0/864796766 conn(0x7fbbec0a9db0 msgr2=0x7fbbec0ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 -- 172.21.15.100:0/864796766 shutdown_connections 2024-08-17T02:22:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 -- 172.21.15.100:0/864796766 wait complete. 2024-08-17T02:22:13.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 Processor -- start 2024-08-17T02:22:13.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 -- start start 2024-08-17T02:22:13.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.636+0000 7fbbf441e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec0b29f0 0x7fbbec14a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf441e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec153a40 0x7fbbec096c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf441e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec15a260 0x7fbbec0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf441e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec14a330 con 0x7fbbec153a40 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf441e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec147c70 con 0x7fbbec15a260 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf441e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec147f10 con 0x7fbbec0b29f0 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec153a40 0x7fbbec096c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec153a40 0x7fbbec096c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56764/0 (socket says 172.21.15.100:56764) 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 -- 172.21.15.100:0/3115138793 learned_addr learned my addr 172.21.15.100:0/3115138793 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf21ba700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec0b29f0 0x7fbbec14a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf29bb700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec15a260 0x7fbbec0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 -- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec0b29f0 msgr2=0x7fbbec14a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec0b29f0 0x7fbbec14a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 -- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec15a260 msgr2=0x7fbbec0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec15a260 0x7fbbec0ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 -- 172.21.15.100:0/3115138793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8009420 con 0x7fbbec153a40 2024-08-17T02:22:13.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf19b9700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec153a40 0x7fbbec096c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00cb70 tx=0x7fbbdc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbe27fc700 1 -- 172.21.15.100:0/3115138793 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc007360 con 0x7fbbec153a40 2024-08-17T02:22:13.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbe27fc700 1 -- 172.21.15.100:0/3115138793 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc0074e0 con 0x7fbbec153a40 2024-08-17T02:22:13.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbec149ce0 con 0x7fbbec153a40 2024-08-17T02:22:13.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbe27fc700 1 -- 172.21.15.100:0/3115138793 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc004830 con 0x7fbbec153a40 2024-08-17T02:22:13.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec0c9640 con 0x7fbbec153a40 2024-08-17T02:22:13.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.640+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec14a4a0 con 0x7fbbec153a40 2024-08-17T02:22:13.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.644+0000 7fbbe27fc700 1 -- 172.21.15.100:0/3115138793 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbbdc004ba0 con 0x7fbbec153a40 2024-08-17T02:22:13.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.644+0000 7fbbe27fc700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd0060860 0x7fbbd0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.644+0000 7fbbe27fc700 1 -- 172.21.15.100:0/3115138793 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbbdc07c430 con 0x7fbbec153a40 2024-08-17T02:22:13.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.644+0000 7fbbf21ba700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd0060860 0x7fbbd0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.644+0000 7fbbf21ba700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd0060860 0x7fbbd0062d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80030c0 tx=0x7fbbe8016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.648+0000 7fbbe27fc700 1 -- 172.21.15.100:0/3115138793 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbbdc04c900 con 0x7fbbec153a40 2024-08-17T02:22:13.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.816+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbec0645c0 con 0x7fbbec153a40 2024-08-17T02:22:13.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.816+0000 7fbbe27fc700 1 -- 172.21.15.100:0/3115138793 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fbbdc018090 con 0x7fbbec153a40 2024-08-17T02:22:13.821 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:13.821 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd0060860 msgr2=0x7fbbd0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd0060860 0x7fbbd0062d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80030c0 tx=0x7fbbe8016000 comp rx=0 tx=0).stop 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec153a40 msgr2=0x7fbbec096c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec153a40 0x7fbbec096c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00cb70 tx=0x7fbbdc001b50 comp rx=0 tx=0).stop 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 shutdown_connections 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd0060860 0x7fbbd0062d10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec15a260 0x7fbbec0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec153a40 0x7fbbec096c20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 --2- 172.21.15.100:0/3115138793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec0b29f0 0x7fbbec14a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 >> 172.21.15.100:0/3115138793 conn(0x7fbbec0a9db0 msgr2=0x7fbbec0ac190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 shutdown_connections 2024-08-17T02:22:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.820+0000 7fbbf441e700 1 -- 172.21.15.100:0/3115138793 wait complete. 2024-08-17T02:22:13.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-08-17T02:22:13.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-08-17T02:22:13.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-08-17T02:22:13.837 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-08-17T02:22:13.838 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-08-17T02:22:13.838 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-08-17T02:22:13.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-08-17T02:22:13.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-08-17T02:22:13.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f48a2a4c700 1 Processor -- start 2024-08-17T02:22:13.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f48a2a4c700 1 -- start start 2024-08-17T02:22:13.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f8631efd700 1 Processor -- start 2024-08-17T02:22:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f48a2a4c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f489c13b630 0x7f489c0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f48a2a4c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 0x7f489c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f48a2a4c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489c13cf90 con 0x7f489c0b1950 2024-08-17T02:22:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f48a2a4c700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489c13c380 con 0x7f489c13b630 2024-08-17T02:22:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f48a2a4c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c13c660 con 0x7f489c13bf10 2024-08-17T02:22:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f489c13b630 0x7f489c0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f489c13b630 0x7f489c0b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56772/0 (socket says 172.21.15.100:56772) 2024-08-17T02:22:13.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489bfff700 1 -- 172.21.15.100:0/1687894970 learned_addr learned my addr 172.21.15.100:0/1687894970 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:13.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f8631efd700 1 -- start start 2024-08-17T02:22:13.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489b7fe700 1 --2- 172.21.15.100:0/1687894970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 0x7f489c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489bfff700 1 -- 172.21.15.100:0/1687894970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 msgr2=0x7f489c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:13.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489bfff700 1 --2- 172.21.15.100:0/1687894970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 0x7f489c06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489bfff700 1 -- 172.21.15.100:0/1687894970 >> v1:172.21.15.123:6789/0 conn(0x7f489c0b1950 legacy=0x7f489c158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:13.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489bfff700 1 -- 172.21.15.100:0/1687894970 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c13c940 con 0x7f489c13b630 2024-08-17T02:22:13.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f8631efd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f862c13b630 0x7f862c0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f8631efd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c13bf10 0x7f862c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f489bfff700 1 --2- 172.21.15.100:0/1687894970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c13b630 0x7f489c0b1410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f489400d1a0 tx=0x7f4894007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1c81364a5da463 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f0db6a3d700 1 Processor -- start 2024-08-17T02:22:13.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f8631efd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c13cf90 con 0x7f862c0b1950 2024-08-17T02:22:13.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f8631efd700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c13c380 con 0x7f862c13b630 2024-08-17T02:22:13.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f8631efd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c13c660 con 0x7f862c13bf10 2024-08-17T02:22:13.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f489affd700 1 -- 172.21.15.100:0/1687894970 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894010070 con 0x7f489c13b630 2024-08-17T02:22:13.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f489affd700 1 -- 172.21.15.100:0/1687894970 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489400dc70 con 0x7f489c13b630 2024-08-17T02:22:13.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f489affd700 1 -- 172.21.15.100:0/1687894970 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894007b40 con 0x7f489c13b630 2024-08-17T02:22:13.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.976+0000 7f0db6a3d700 1 -- start start 2024-08-17T02:22:13.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862affd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c13bf10 0x7f862c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f862c0b1950 0x7f862c158db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:54550/0 (socket says 172.21.15.100:54550) 2024-08-17T02:22:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862bfff700 1 -- 172.21.15.100:0/3224914062 learned_addr learned my addr 172.21.15.100:0/3224914062 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862affd700 1 -- 172.21.15.100:0/3224914062 >> v2:172.21.15.100:3300/0 conn(0x7f862c13b630 msgr2=0x7f862c0b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862a7fc700 1 -- 172.21.15.100:0/3224914062 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679154366 0 0) 0x7f862c13cf90 con 0x7f862c0b1950 2024-08-17T02:22:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862b7fe700 1 --2- 172.21.15.100:0/3224914062 >> v2:172.21.15.100:3300/0 conn(0x7f862c13b630 0x7f862c0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 -- 172.21.15.100:0/1687894970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c13b630 msgr2=0x7f489c0b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 --2- 172.21.15.100:0/1687894970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c13b630 0x7f489c0b1410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f489400d1a0 tx=0x7f4894007690 comp rx=0 tx=0).stop 2024-08-17T02:22:13.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862affd700 1 --2- 172.21.15.100:0/3224914062 >> v2:172.21.15.100:3300/0 conn(0x7f862c13b630 0x7f862c0b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862affd700 1 -- 172.21.15.100:0/3224914062 >> v1:172.21.15.123:6789/0 conn(0x7f862c0b1950 legacy=0x7f862c158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862affd700 1 -- 172.21.15.100:0/3224914062 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c13c940 con 0x7f862c13bf10 2024-08-17T02:22:13.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 -- 172.21.15.100:0/1687894970 shutdown_connections 2024-08-17T02:22:13.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 --2- 172.21.15.100:0/1687894970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 0x7f489c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 --2- 172.21.15.100:0/1687894970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c13b630 0x7f489c0b1410 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 -- 172.21.15.100:0/1687894970 >> 172.21.15.100:0/1687894970 conn(0x7f489c0678d0 msgr2=0x7f489c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:13.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 -- 172.21.15.100:0/1687894970 shutdown_connections 2024-08-17T02:22:13.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 -- 172.21.15.100:0/1687894970 wait complete. 2024-08-17T02:22:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0db6a3d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 0x7f0db00b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862b7fe700 1 --2- 172.21.15.100:0/3224914062 >> v2:172.21.15.100:3300/0 conn(0x7f862c13b630 0x7f862c0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0db6a3d700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db013cf90 con 0x7f0db00b1950 2024-08-17T02:22:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0db6a3d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db013c380 con 0x7f0db013bf10 2024-08-17T02:22:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0db6a3d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db013c660 con 0x7f0db013b630 2024-08-17T02:22:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862affd700 1 --2- 172.21.15.100:0/3224914062 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c13bf10 0x7f862c06df10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f861c0098e0 tx=0x7f861c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a754bbbf3a93bb0a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0db4fda700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0db00b1950 0x7f0db013d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47016/0 (socket says 172.21.15.100:47016) 2024-08-17T02:22:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0db4fda700 1 -- 172.21.15.100:0/4203840658 learned_addr learned my addr 172.21.15.100:0/4203840658 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0daffff700 1 --2- 172.21.15.100:0/4203840658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 0x7f0db00b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:13.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862a7fc700 1 -- 172.21.15.100:0/3224914062 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c009e80 con 0x7f862c13bf10 2024-08-17T02:22:13.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0daeffd700 1 -- 172.21.15.100:0/4203840658 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256102154 0 0) 0x7f0db013cf90 con 0x7f0db00b1950 2024-08-17T02:22:13.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0daeffd700 1 -- 172.21.15.100:0/4203840658 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d98003570 con 0x7f0db00b1950 2024-08-17T02:22:13.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862a7fc700 1 -- 172.21.15.100:0/3224914062 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c00b860 con 0x7f862c13bf10 2024-08-17T02:22:13.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f862a7fc700 1 -- 172.21.15.100:0/3224914062 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c00bc00 con 0x7f862c13bf10 2024-08-17T02:22:13.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0daffff700 1 -- 172.21.15.100:0/4203840658 >> v1:172.21.15.123:6789/0 conn(0x7f0db013bf10 legacy=0x7f0db006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0daffff700 1 -- 172.21.15.100:0/4203840658 >> v1:172.21.15.100:6789/0 conn(0x7f0db00b1950 legacy=0x7f0db013d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0daffff700 1 -- 172.21.15.100:0/4203840658 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db013c940 con 0x7f0db013b630 2024-08-17T02:22:13.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f3292bb5700 1 Processor -- start 2024-08-17T02:22:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0daffff700 1 --2- 172.21.15.100:0/4203840658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 0x7f0db00b1410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0da800afa0 tx=0x7f0da800a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9b86b1b0e48106 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f0daeffd700 1 -- 172.21.15.100:0/4203840658 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da800f070 con 0x7f0db013b630 2024-08-17T02:22:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7efd999a2700 1 Processor -- start 2024-08-17T02:22:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7efd999a2700 1 -- start start 2024-08-17T02:22:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f8631efd700 1 -- 172.21.15.100:0/3224914062 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c13bf10 msgr2=0x7f862c06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f8631efd700 1 --2- 172.21.15.100:0/3224914062 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c13bf10 0x7f862c06df10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f861c0098e0 tx=0x7f861c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f8631efd700 1 -- 172.21.15.100:0/3224914062 shutdown_connections 2024-08-17T02:22:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f8631efd700 1 --2- 172.21.15.100:0/3224914062 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c13bf10 0x7f862c06df10 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f861c0098e0 tx=0x7f861c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f8631efd700 1 --2- 172.21.15.100:0/3224914062 >> v2:172.21.15.100:3300/0 conn(0x7f862c13b630 0x7f862c0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f8631efd700 1 -- 172.21.15.100:0/3224914062 >> 172.21.15.100:0/3224914062 conn(0x7f862c0678d0 msgr2=0x7f862c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f8631efd700 1 -- 172.21.15.100:0/3224914062 shutdown_connections 2024-08-17T02:22:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 Processor -- start 2024-08-17T02:22:13.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f3292bb5700 1 -- start start 2024-08-17T02:22:13.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f8631efd700 1 -- 172.21.15.100:0/3224914062 wait complete. 2024-08-17T02:22:13.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f48a2a4c700 1 -- start start 2024-08-17T02:22:14.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f3292bb5700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32840dfcf0 0x7f32840e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f3292bb5700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3284055050 con 0x7f328400e090 2024-08-17T02:22:14.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f3292bb5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328400cbb0 con 0x7f32840e3b70 2024-08-17T02:22:14.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.980+0000 7f3292bb5700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32840e46e0 con 0x7f32840dfcf0 2024-08-17T02:22:14.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3291152700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f328400e090 0x7f32840554e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47026/0 (socket says 172.21.15.100:47026) 2024-08-17T02:22:14.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3291152700 1 -- 172.21.15.100:0/2541812029 learned_addr learned my addr 172.21.15.100:0/2541812029 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:14.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3290951700 1 --2- 172.21.15.100:0/2541812029 >> v2:172.21.15.100:3300/0 conn(0x7f32840dfcf0 0x7f32840e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0daeffd700 1 -- 172.21.15.100:0/4203840658 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da800a2e0 con 0x7f0db013b630 2024-08-17T02:22:14.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0daeffd700 1 -- 172.21.15.100:0/4203840658 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da800cc60 con 0x7f0db013b630 2024-08-17T02:22:14.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f2812f1e700 1 Processor -- start 2024-08-17T02:22:14.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd999a2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd940a9430 0x7efd940ab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd999a2700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd940af160 con 0x7efd9406dbe0 2024-08-17T02:22:14.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd999a2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9406e330 con 0x7efd941428f0 2024-08-17T02:22:14.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd999a2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd940ab740 con 0x7efd940a9430 2024-08-17T02:22:14.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f328b7fe700 1 -- 172.21.15.100:0/2541812029 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235841397 0 0) 0x7f3284055050 con 0x7f328400e090 2024-08-17T02:22:14.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f8631efd700 1 Processor -- start 2024-08-17T02:22:14.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f2812f1e700 1 -- start start 2024-08-17T02:22:14.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd989a0700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efd941428f0 0x7efd940a8d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:54582/0 (socket says 172.21.15.100:54582) 2024-08-17T02:22:14.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd989a0700 1 -- 172.21.15.100:0/1021009226 learned_addr learned my addr 172.21.15.100:0/1021009226 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:14.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3290951700 1 -- 172.21.15.100:0/2541812029 >> v1:172.21.15.123:6789/0 conn(0x7f32840e3b70 legacy=0x7f328400d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3290951700 1 -- 172.21.15.100:0/2541812029 >> v1:172.21.15.100:6789/0 conn(0x7f328400e090 legacy=0x7f32840554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3290951700 1 -- 172.21.15.100:0/2541812029 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840e49c0 con 0x7f32840dfcf0 2024-08-17T02:22:14.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd93fff700 1 --2- 172.21.15.100:0/1021009226 >> v2:172.21.15.123:3300/0 conn(0x7efd940a9430 0x7efd940ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a2a4c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489c0b1950 0x7f489c158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a2a4c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c13b630 0x7f489c154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a2a4c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 0x7f489c154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a2a4c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c158e00 con 0x7f489c13b630 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a2a4c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c16e1d0 con 0x7f489c0b1950 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a2a4c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c158b20 con 0x7f489c13bf10 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a0fe9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 0x7f489c154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f489bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489c0b1950 0x7f489c158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a0fe9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 0x7f489c154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:51654/0 (socket says 172.21.15.100:51654) 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a0fe9700 1 -- 172.21.15.100:0/4039682757 learned_addr learned my addr 172.21.15.100:0/4039682757 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f489b7fe700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c13b630 0x7f489c154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f8631efd700 1 -- start start 2024-08-17T02:22:14.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd937fe700 1 -- 172.21.15.100:0/1021009226 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889580358 0 0) 0x7efd940af160 con 0x7efd9406dbe0 2024-08-17T02:22:14.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd937fe700 1 -- 172.21.15.100:0/1021009226 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd7c003570 con 0x7efd9406dbe0 2024-08-17T02:22:14.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c672de700 1 Processor -- start 2024-08-17T02:22:14.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd937fe700 1 -- 172.21.15.100:0/1021009226 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889552545 0 0) 0x7efd9406e330 con 0x7efd941428f0 2024-08-17T02:22:14.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd937fe700 1 -- 172.21.15.100:0/1021009226 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd940af160 con 0x7efd941428f0 2024-08-17T02:22:14.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0db6a3d700 1 -- 172.21.15.100:0/4203840658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 msgr2=0x7f0db00b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0db6a3d700 1 --2- 172.21.15.100:0/4203840658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 0x7f0db00b1410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0da800afa0 tx=0x7f0da800a890 comp rx=0 tx=0).stop 2024-08-17T02:22:14.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c672de700 1 -- start start 2024-08-17T02:22:14.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f489bfff700 1 -- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 msgr2=0x7f489c154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f489bfff700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489c13bf10 0x7f489c154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f489bfff700 1 -- 172.21.15.100:0/4039682757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c13b630 msgr2=0x7f489c154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f489bfff700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c13b630 0x7f489c154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f489bfff700 1 -- 172.21.15.100:0/4039682757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489400be70 con 0x7f489c0b1950 2024-08-17T02:22:14.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0db6a3d700 1 -- 172.21.15.100:0/4203840658 shutdown_connections 2024-08-17T02:22:14.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0db6a3d700 1 --2- 172.21.15.100:0/4203840658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 0x7f0db00b1410 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0db6a3d700 1 -- 172.21.15.100:0/4203840658 >> 172.21.15.100:0/4203840658 conn(0x7f0db00678d0 msgr2=0x7f0db0067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f2812f1e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f280c06e7d0 0x7f280c146690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd937fe700 1 -- 172.21.15.100:0/1021009226 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 817211796 0 0) 0x7efd7c003570 con 0x7efd9406dbe0 2024-08-17T02:22:14.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd93fff700 1 -- 172.21.15.100:0/1021009226 >> v1:172.21.15.123:6789/0 conn(0x7efd941428f0 legacy=0x7efd940a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd93fff700 1 -- 172.21.15.100:0/1021009226 >> v1:172.21.15.100:6789/0 conn(0x7efd9406dbe0 legacy=0x7efd940af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3290951700 1 --2- 172.21.15.100:0/2541812029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32840dfcf0 0x7f32840e00c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f32780098e0 tx=0x7f32780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4573bbeecca0ac47 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0db6a3d700 1 -- 172.21.15.100:0/4203840658 shutdown_connections 2024-08-17T02:22:14.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f489bfff700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489c0b1950 0x7f489c158290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4894017000 tx=0x7f489401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f2812f1e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280c15da10 con 0x7f280c146bd0 2024-08-17T02:22:14.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f28114bb700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f280c146bd0 0x7f280c144930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47050/0 (socket says 172.21.15.100:47050) 2024-08-17T02:22:14.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f28114bb700 1 -- 172.21.15.100:0/4047880253 learned_addr learned my addr 172.21.15.100:0/4047880253 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:14.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f328b7fe700 1 -- 172.21.15.100:0/2541812029 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278009e80 con 0x7f32840dfcf0 2024-08-17T02:22:14.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f2812f1e700 1 -- 172.21.15.100:0/4047880253 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280c1444a0 con 0x7f280c06e400 2024-08-17T02:22:14.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f2812f1e700 1 -- 172.21.15.100:0/4047880253 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c147020 con 0x7f280c06e7d0 2024-08-17T02:22:14.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f328b7fe700 1 -- 172.21.15.100:0/2541812029 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327800b860 con 0x7f32840dfcf0 2024-08-17T02:22:14.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f280bfff700 1 --2- 172.21.15.100:0/4047880253 >> v2:172.21.15.100:3300/0 conn(0x7f280c06e7d0 0x7f280c146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c672de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006e140 0x7f7c6006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c672de700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7c6006aff0 0x7f7c601556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c672de700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c60156940 con 0x7f7c600b1660 2024-08-17T02:22:14.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c672de700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c60155d30 con 0x7f7c6006aff0 2024-08-17T02:22:14.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c672de700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c60156010 con 0x7f7c6006e140 2024-08-17T02:22:14.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f8631efd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f862c0b1950 0x7f862c13a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f8631efd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f862c13b630 0x7f862c163930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f8631efd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c163e70 0x7f862c0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400de60 con 0x7f489c0b1950 2024-08-17T02:22:14.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4894020460 con 0x7f489c0b1950 2024-08-17T02:22:14.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894020820 con 0x7f489c0b1950 2024-08-17T02:22:14.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c16db80 con 0x7f489c0b1950 2024-08-17T02:22:14.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c1587d0 con 0x7f489c0b1950 2024-08-17T02:22:14.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f0db6a3d700 1 -- 172.21.15.100:0/4203840658 wait complete. 2024-08-17T02:22:14.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f280bfff700 1 -- 172.21.15.100:0/4047880253 >> v1:172.21.15.123:6789/0 conn(0x7f280c06e400 legacy=0x7f280c0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f280bfff700 1 -- 172.21.15.100:0/4047880253 >> v1:172.21.15.100:6789/0 conn(0x7f280c146bd0 legacy=0x7f280c144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f280bfff700 1 -- 172.21.15.100:0/4047880253 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c144170 con 0x7f280c06e7d0 2024-08-17T02:22:14.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd93fff700 1 -- 172.21.15.100:0/1021009226 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9406e610 con 0x7efd940a9430 2024-08-17T02:22:14.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7efd93fff700 1 --2- 172.21.15.100:0/1021009226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd940a9430 0x7efd940ab200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009de0 tx=0x7efd8c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce49ce416611680 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f328b7fe700 1 -- 172.21.15.100:0/2541812029 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800bbe0 con 0x7f32840dfcf0 2024-08-17T02:22:14.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f8631efd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1642d0 con 0x7f862c13b630 2024-08-17T02:22:14.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f8631efd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0cb4a0 con 0x7f862c0b1950 2024-08-17T02:22:14.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f8631efd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0ca890 con 0x7f862c163e70 2024-08-17T02:22:14.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f862c13b630 0x7f862c163930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f862c13b630 0x7f862c163930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56822/0 (socket says 172.21.15.100:56822) 2024-08-17T02:22:14.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 -- 172.21.15.100:0/1528768808 learned_addr learned my addr 172.21.15.100:0/1528768808 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862b7fe700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f862c0b1950 0x7f862c13a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862bfff700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c163e70 0x7f862c0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 -- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c163e70 msgr2=0x7f862c0c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f862c163e70 0x7f862c0c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 -- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f862c0b1950 msgr2=0x7f862c13a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f862c0b1950 0x7f862c13a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 -- 172.21.15.100:0/1528768808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c009420 con 0x7f862c13b630 2024-08-17T02:22:14.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3292bb5700 1 -- 172.21.15.100:0/2541812029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32840dfcf0 msgr2=0x7f32840e00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3292bb5700 1 --2- 172.21.15.100:0/2541812029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32840dfcf0 0x7f32840e00c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f32780098e0 tx=0x7f32780091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3292bb5700 1 -- 172.21.15.100:0/2541812029 shutdown_connections 2024-08-17T02:22:14.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3292bb5700 1 --2- 172.21.15.100:0/2541812029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32840dfcf0 0x7f32840e00c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f3292bb5700 1 -- 172.21.15.100:0/2541812029 >> 172.21.15.100:0/2541812029 conn(0x7f3284057900 msgr2=0x7f328400c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f862affd700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f862c13b630 0x7f862c163930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f861c013000 tx=0x7f861c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f752e474700 1 Processor -- start 2024-08-17T02:22:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f280bfff700 1 --2- 172.21.15.100:0/4047880253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e7d0 0x7f280c146690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f28000098e0 tx=0x7f28000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a02332524d162dc server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd937fe700 1 -- 172.21.15.100:0/1021009226 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00e070 con 0x7efd940a9430 2024-08-17T02:22:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f86137fe700 1 -- 172.21.15.100:0/1528768808 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c00bbe0 con 0x7f862c13b630 2024-08-17T02:22:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f86137fe700 1 -- 172.21.15.100:0/1528768808 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c011420 con 0x7f862c13b630 2024-08-17T02:22:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f86137fe700 1 -- 172.21.15.100:0/1528768808 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c0117c0 con 0x7f862c13b630 2024-08-17T02:22:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c0cab70 con 0x7f862c13b630 2024-08-17T02:22:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c0cb130 con 0x7f862c13b630 2024-08-17T02:22:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f489c16d9e0 con 0x7f489c0b1950 2024-08-17T02:22:14.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0db6a3d700 1 Processor -- start 2024-08-17T02:22:14.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0db6a3d700 1 -- start start 2024-08-17T02:22:14.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f752e474700 1 -- start start 2024-08-17T02:22:14.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd937fe700 1 -- 172.21.15.100:0/1021009226 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c00b7a0 con 0x7efd940a9430 2024-08-17T02:22:14.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd937fe700 1 -- 172.21.15.100:0/1021009226 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88001470 con 0x7efd940a9430 2024-08-17T02:22:14.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd999a2700 1 -- 172.21.15.100:0/1021009226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd940a9430 msgr2=0x7efd940ab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd999a2700 1 --2- 172.21.15.100:0/1021009226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd940a9430 0x7efd940ab200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009de0 tx=0x7efd8c009780 comp rx=0 tx=0).stop 2024-08-17T02:22:14.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f3292bb5700 1 -- 172.21.15.100:0/2541812029 shutdown_connections 2024-08-17T02:22:14.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f752e474700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f75281592f0 0x7f752806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0db6a3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db00b1950 0x7f0db016a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0db6a3d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 0x7f0db00ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0db6a3d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db013bf10 0x7f0db00cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f752e474700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75280aed60 con 0x7f752806e450 2024-08-17T02:22:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f752e474700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752806e850 con 0x7f7528158f20 2024-08-17T02:22:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f752e474700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75280a9f70 con 0x7f75281592f0 2024-08-17T02:22:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c65adb700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7c6006aff0 0x7f7c601556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c65adb700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7c6006aff0 0x7f7c601556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56830/0 (socket says 172.21.15.100:56830) 2024-08-17T02:22:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.984+0000 7f7c65adb700 1 -- 172.21.15.100:0/3200987054 learned_addr learned my addr 172.21.15.100:0/3200987054 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c662dc700 1 --2- 172.21.15.100:0/3200987054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006e140 0x7f7c6006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c65adb700 1 -- 172.21.15.100:0/3200987054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006e140 msgr2=0x7f7c6006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c65adb700 1 --2- 172.21.15.100:0/3200987054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006e140 0x7f7c6006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c65adb700 1 -- 172.21.15.100:0/3200987054 >> v1:172.21.15.100:6789/0 conn(0x7f7c600b1660 legacy=0x7f7c6013e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c65adb700 1 -- 172.21.15.100:0/3200987054 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c601562f0 con 0x7f7c6006aff0 2024-08-17T02:22:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c662dc700 1 --2- 172.21.15.100:0/3200987054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006e140 0x7f7c6006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0db6a3d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db0169a90 con 0x7f0db013bf10 2024-08-17T02:22:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0db6a3d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db006e760 con 0x7f0db00b1950 2024-08-17T02:22:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0db6a3d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db016a6a0 con 0x7f0db013b630 2024-08-17T02:22:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7527fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7528158f20 0x7f75280a9860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:54604/0 (socket says 172.21.15.100:54604) 2024-08-17T02:22:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7527fff700 1 -- 172.21.15.100:0/2757358998 learned_addr learned my addr 172.21.15.100:0/2757358998 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c65adb700 1 --2- 172.21.15.100:0/3200987054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6006aff0 0x7f7c601556f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c010c90 tx=0x7f7c5c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cde2112c3230a3a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db00b1950 0x7f0db016a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db00b1950 0x7f0db016a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:45798/0 (socket says 172.21.15.100:45798) 2024-08-17T02:22:14.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 -- 172.21.15.100:0/2719135166 learned_addr learned my addr 172.21.15.100:0/2719135166 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daf7fe700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 0x7f0db00ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c652da700 1 -- 172.21.15.100:0/3200987054 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0100d0 con 0x7f7c6006aff0 2024-08-17T02:22:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c652da700 1 -- 172.21.15.100:0/3200987054 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c010230 con 0x7f7c6006aff0 2024-08-17T02:22:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c652da700 1 -- 172.21.15.100:0/3200987054 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0105f0 con 0x7f7c6006aff0 2024-08-17T02:22:14.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f280b7fe700 1 -- 172.21.15.100:0/4047880253 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800009e80 con 0x7f280c06e7d0 2024-08-17T02:22:14.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f280b7fe700 1 -- 172.21.15.100:0/4047880253 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280000b820 con 0x7f280c06e7d0 2024-08-17T02:22:14.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f280b7fe700 1 -- 172.21.15.100:0/4047880253 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000bbc0 con 0x7f280c06e7d0 2024-08-17T02:22:14.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f2812f1e700 1 -- 172.21.15.100:0/4047880253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e7d0 msgr2=0x7f280c146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f2812f1e700 1 --2- 172.21.15.100:0/4047880253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e7d0 0x7f280c146690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f28000098e0 tx=0x7f28000091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 -- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 msgr2=0x7f0db00ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0db013b630 0x7f0db00ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 -- 172.21.15.100:0/2719135166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db013bf10 msgr2=0x7f0db00cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db013bf10 0x7f0db00cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 -- 172.21.15.100:0/2719135166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da800ab30 con 0x7f0db00b1950 2024-08-17T02:22:14.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f3292bb5700 1 -- 172.21.15.100:0/2541812029 wait complete. 2024-08-17T02:22:14.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7526ffd700 1 -- 172.21.15.100:0/2757358998 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929439331 0 0) 0x7f752806e850 con 0x7f7528158f20 2024-08-17T02:22:14.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7526ffd700 1 -- 172.21.15.100:0/2757358998 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7510003570 con 0x7f7528158f20 2024-08-17T02:22:14.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f75277fe700 1 --2- 172.21.15.100:0/2757358998 >> v2:172.21.15.100:3300/0 conn(0x7f75281592f0 0x7f752806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7526ffd700 1 -- 172.21.15.100:0/2757358998 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145579556 0 0) 0x7f75280aed60 con 0x7f752806e450 2024-08-17T02:22:14.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7526ffd700 1 -- 172.21.15.100:0/2757358998 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752806e850 con 0x7f752806e450 2024-08-17T02:22:14.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c672de700 1 -- 172.21.15.100:0/3200987054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6006aff0 msgr2=0x7f7c601556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c672de700 1 --2- 172.21.15.100:0/3200987054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6006aff0 0x7f7c601556f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c010c90 tx=0x7f7c5c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd999a2700 1 -- 172.21.15.100:0/1021009226 shutdown_connections 2024-08-17T02:22:14.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd999a2700 1 --2- 172.21.15.100:0/1021009226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd940a9430 0x7efd940ab200 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd999a2700 1 -- 172.21.15.100:0/1021009226 >> 172.21.15.100:0/1021009226 conn(0x7efd94066e00 msgr2=0x7efd94067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd999a2700 1 -- 172.21.15.100:0/1021009226 shutdown_connections 2024-08-17T02:22:14.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f86137fe700 1 -- 172.21.15.100:0/1528768808 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f861c011bb0 con 0x7f862c13b630 2024-08-17T02:22:14.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f86137fe700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f86180608e0 0x7f8618062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f86137fe700 1 -- 172.21.15.100:0/1528768808 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f861c07e540 con 0x7f862c13b630 2024-08-17T02:22:14.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c672de700 1 -- 172.21.15.100:0/3200987054 shutdown_connections 2024-08-17T02:22:14.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c672de700 1 --2- 172.21.15.100:0/3200987054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6006aff0 0x7f7c601556f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c672de700 1 --2- 172.21.15.100:0/3200987054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006e140 0x7f7c6006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f8631efd700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f862c06a3d0 0x7f862c0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f75277fe700 1 -- 172.21.15.100:0/2757358998 >> v1:172.21.15.123:6789/0 conn(0x7f7528158f20 legacy=0x7f75280a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f75277fe700 1 -- 172.21.15.100:0/2757358998 >> v1:172.21.15.100:6789/0 conn(0x7f752806e450 legacy=0x7f75280ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f3292bb5700 1 Processor -- start 2024-08-17T02:22:14.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c672de700 1 -- 172.21.15.100:0/3200987054 >> 172.21.15.100:0/3200987054 conn(0x7f7c600678d0 msgr2=0x7f7c60067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f75277fe700 1 -- 172.21.15.100:0/2757358998 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280aa240 con 0x7f75281592f0 2024-08-17T02:22:14.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f48940153a0 con 0x7f489c0b1950 2024-08-17T02:22:14.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f4883fff700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4888060860 0x7f4888062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f489407ee60 con 0x7f489c0b1950 2024-08-17T02:22:14.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 --> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f862c0ca660 con 0x7f862c06a3d0 2024-08-17T02:22:14.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f4883fff700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7f4888068a70 0x7f488806ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 --> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f488806b530 con 0x7f4888068a70 2024-08-17T02:22:14.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f489407f150 con 0x7f489c0b1950 2024-08-17T02:22:14.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f2812f1e700 1 -- 172.21.15.100:0/4047880253 shutdown_connections 2024-08-17T02:22:14.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f2812f1e700 1 --2- 172.21.15.100:0/4047880253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e7d0 0x7f280c146690 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f2812f1e700 1 -- 172.21.15.100:0/4047880253 >> 172.21.15.100:0/4047880253 conn(0x7f280c067890 msgr2=0x7f280c067c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f2812f1e700 1 -- 172.21.15.100:0/4047880253 shutdown_connections 2024-08-17T02:22:14.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f2812f1e700 1 -- 172.21.15.100:0/4047880253 wait complete. 2024-08-17T02:22:14.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f0daffff700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db00b1950 0x7f0db016a160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0da800a8d0 tx=0x7f0da8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f3292bb5700 1 -- start start 2024-08-17T02:22:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f862b7fe700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f86180608e0 0x7f8618062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f7c672de700 1 -- 172.21.15.100:0/3200987054 shutdown_connections 2024-08-17T02:22:14.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f862bfff700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f862c06a3d0 0x7f862c0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7f48a0fe9700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7f4888068a70 0x7f488806ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f489b7fe700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4888060860 0x7f4888062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.988+0000 7efd999a2700 1 -- 172.21.15.100:0/1021009226 wait complete. 2024-08-17T02:22:14.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c672de700 1 -- 172.21.15.100:0/3200987054 wait complete. 2024-08-17T02:22:14.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f48a0fe9700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7f4888068a70 0x7f488806ae80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f328400e090 0x7f328403b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32840dfcf0 0x7f3284050630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32840e3b70 0x7f32840f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328403ad70 con 0x7f32840dfcf0 2024-08-17T02:22:14.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328403a160 con 0x7f328400e090 2024-08-17T02:22:14.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328403a440 con 0x7f32840e3b70 2024-08-17T02:22:14.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f489b7fe700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4888060860 0x7f4888062d10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f489c06e0f0 tx=0x7f488c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3290951700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f328400e090 0x7f328403b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3290951700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f328400e090 0x7f328403b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:45800/0 (socket says 172.21.15.100:45800) 2024-08-17T02:22:14.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3290951700 1 -- 172.21.15.100:0/3131254803 learned_addr learned my addr 172.21.15.100:0/3131254803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3291152700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32840e3b70 0x7f32840f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f75277fe700 1 --2- 172.21.15.100:0/2757358998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75281592f0 0x7f752806df10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098e0 tx=0x7f751c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8077243c1a341bd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd999a2700 1 Processor -- start 2024-08-17T02:22:14.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2812f1e700 1 Processor -- start 2024-08-17T02:22:14.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3291152700 1 -- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f328400e090 msgr2=0x7f328403b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2812f1e700 1 -- start start 2024-08-17T02:22:14.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c672de700 1 Processor -- start 2024-08-17T02:22:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 <== osd.2 v2:172.21.15.100:6809/24272 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f488806b530 con 0x7f4888068a70 2024-08-17T02:22:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8009a50 con 0x7f0db00b1950 2024-08-17T02:22:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f862bfff700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f862c06a3d0 0x7f862c0c7ce0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f862b7fe700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f86180608e0 0x7f8618062d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8620006fd0 tx=0x7f8620008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f86137fe700 1 -- 172.21.15.100:0/1528768808 <== osd.1 v2:172.21.15.100:6824/24273 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f862c0ca660 con 0x7f862c06a3d0 2024-08-17T02:22:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da8009bb0 con 0x7f0db00b1950 2024-08-17T02:22:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3291152700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f328400e090 0x7f328403b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c672de700 1 -- start start 2024-08-17T02:22:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3291152700 1 -- 172.21.15.100:0/3131254803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32840dfcf0 msgr2=0x7f3284050630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3291152700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32840dfcf0 0x7f3284050630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3291152700 1 -- 172.21.15.100:0/3131254803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3278009420 con 0x7f32840e3b70 2024-08-17T02:22:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3290951700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f328400e090 0x7f328403b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f0db6a3d700 1 -- 172.21.15.100:0/2719135166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db006db50 con 0x7f0db00b1950 2024-08-17T02:22:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8027410 con 0x7f0db00b1950 2024-08-17T02:22:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f0db6a3d700 1 -- 172.21.15.100:0/2719135166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db006e110 con 0x7f0db00b1950 2024-08-17T02:22:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd999a2700 1 -- start start 2024-08-17T02:22:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd999a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9406dbe0 0x7efd9415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd999a2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd940a9430 0x7efd940c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd999a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd941428f0 0x7efd940c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd999a2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940cb3a0 con 0x7efd9406dbe0 2024-08-17T02:22:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd999a2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9415be50 con 0x7efd941428f0 2024-08-17T02:22:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd999a2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd940ca790 con 0x7efd940a9430 2024-08-17T02:22:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3291152700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32840e3b70 0x7f32840f3ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f328c02b300 tx=0x7f328c065310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c0708e0 con 0x7f32840e3b70 2024-08-17T02:22:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c08b070 con 0x7f32840e3b70 2024-08-17T02:22:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c070cb0 con 0x7f32840e3b70 2024-08-17T02:22:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 -- 172.21.15.100:0/3131254803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328403a720 con 0x7f32840e3b70 2024-08-17T02:22:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 -- 172.21.15.100:0/3131254803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3284039df0 con 0x7f32840e3b70 2024-08-17T02:22:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c672de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006aff0 0x7f7c6013a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c672de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6006e140 0x7f7c601367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c672de700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c600b1660 0x7f7c60136cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c66add700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c600b1660 0x7f7c60136cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c672de700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c60138f40 con 0x7f7c600b1660 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c66add700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c600b1660 0x7f7c60136cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56882/0 (socket says 172.21.15.100:56882) 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c66add700 1 -- 172.21.15.100:0/3672744040 learned_addr learned my addr 172.21.15.100:0/3672744040 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2812f1e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e400 0x7f280c143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2812f1e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280c06e7d0 0x7f280c1436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2812f1e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f280c146bd0 0x7f280c170ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2812f1e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c141900 con 0x7f280c06e400 2024-08-17T02:22:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2812f1e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c140cf0 con 0x7f280c06e7d0 2024-08-17T02:22:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2812f1e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c140fd0 con 0x7f280c146bd0 2024-08-17T02:22:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f28114bb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f280c146bd0 0x7f280c170ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f28114bb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f280c146bd0 0x7f280c170ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:51702/0 (socket says 172.21.15.100:51702) 2024-08-17T02:22:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f28114bb700 1 -- 172.21.15.100:0/1567953067 learned_addr learned my addr 172.21.15.100:0/1567953067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2810cba700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e400 0x7f280c143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7526ffd700 1 -- 172.21.15.100:0/2757358998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c011630 con 0x7f75281592f0 2024-08-17T02:22:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7526ffd700 1 -- 172.21.15.100:0/2757358998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c011790 con 0x7f75281592f0 2024-08-17T02:22:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7526ffd700 1 -- 172.21.15.100:0/2757358998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c011b30 con 0x7f75281592f0 2024-08-17T02:22:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f752e474700 1 -- 172.21.15.100:0/2757358998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75281592f0 msgr2=0x7f752806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f752e474700 1 --2- 172.21.15.100:0/2757358998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75281592f0 0x7f752806df10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098e0 tx=0x7f751c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd989a0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9406dbe0 0x7efd9415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd989a0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9406dbe0 0x7efd9415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56868/0 (socket says 172.21.15.100:56868) 2024-08-17T02:22:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd989a0700 1 -- 172.21.15.100:0/1330963715 learned_addr learned my addr 172.21.15.100:0/1330963715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd93fff700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd940a9430 0x7efd940c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd989a0700 1 -- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd940a9430 msgr2=0x7efd940c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd989a0700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd940a9430 0x7efd940c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd989a0700 1 -- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd941428f0 msgr2=0x7efd940c9ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-17T02:22:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd989a0700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd941428f0 0x7efd940c9ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7efd989a0700 1 -- 172.21.15.100:0/1330963715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c009a20 con 0x7efd9406dbe0 2024-08-17T02:22:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f7c662dc700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006aff0 0x7f7c6013a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2810cba700 1 -- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f280c146bd0 msgr2=0x7f280c170ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2810cba700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f280c146bd0 0x7f280c170ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2810cba700 1 -- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280c06e7d0 msgr2=0x7f280c1436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2810cba700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280c06e7d0 0x7f280c1436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f2810cba700 1 -- 172.21.15.100:0/1567953067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800009420 con 0x7f280c06e400 2024-08-17T02:22:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c65adb700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6006e140 0x7f7c601367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c60137670 con 0x7f7c6006e140 2024-08-17T02:22:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c60137390 con 0x7f7c6006aff0 2024-08-17T02:22:14.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f2810cba700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e400 0x7f280c143190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f280400bb00 tx=0x7f2804009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0db6a3d700 1 -- 172.21.15.100:0/2719135166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0d90000fd0 con 0x7f0db00b1950 2024-08-17T02:22:14.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f2808ff9700 1 -- 172.21.15.100:0/1567953067 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280400c840 con 0x7f280c06e400 2024-08-17T02:22:14.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f2808ff9700 1 -- 172.21.15.100:0/1567953067 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280400c9a0 con 0x7f280c06e400 2024-08-17T02:22:14.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c1412b0 con 0x7f280c06e400 2024-08-17T02:22:14.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f752e474700 1 -- 172.21.15.100:0/2757358998 shutdown_connections 2024-08-17T02:22:14.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f752e474700 1 --2- 172.21.15.100:0/2757358998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75281592f0 0x7f752806df10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f752e474700 1 -- 172.21.15.100:0/2757358998 >> 172.21.15.100:0/2757358998 conn(0x7f75280678d0 msgr2=0x7f7528067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.992+0000 7f3292bb5700 1 -- 172.21.15.100:0/3131254803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3284005800 con 0x7f32840e3b70 2024-08-17T02:22:14.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f328c06f150 con 0x7f32840e3b70 2024-08-17T02:22:14.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7efd989a0700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9406dbe0 0x7efd9415b910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd84007fb0 tx=0x7efd84007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c66add700 1 -- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006aff0 msgr2=0x7f7c6013a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c66add700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006aff0 0x7f7c6013a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c66add700 1 -- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6006e140 msgr2=0x7f7c601367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c66add700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6006e140 0x7f7c601367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c66add700 1 -- 172.21.15.100:0/3672744040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c0107d0 con 0x7f7c600b1660 2024-08-17T02:22:14.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f3288ff9700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3270060860 0x7f3270062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0da8027780 con 0x7f0db00b1950 2024-08-17T02:22:14.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f328c0f3bc0 con 0x7f32840e3b70 2024-08-17T02:22:14.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f3288ff9700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f32700689f0 0x7f327006ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 --> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f327006b4b0 con 0x7f32700689f0 2024-08-17T02:22:14.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f3290951700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3270060860 0x7f3270062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f328c0f3f90 con 0x7f32840e3b70 2024-08-17T02:22:14.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0d97fff700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0d9c0609b0 0x7f0d9c062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0da807e890 con 0x7f0db00b1950 2024-08-17T02:22:14.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0d97fff700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f0d9c068bc0 0x7f0d9c06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 --> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d9c06b680 con 0x7f0d9c068bc0 2024-08-17T02:22:14.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0da8083220 con 0x7f0db00b1950 2024-08-17T02:22:14.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f2808ff9700 1 -- 172.21.15.100:0/1567953067 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280400cd60 con 0x7f280c06e400 2024-08-17T02:22:14.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c14d5e0 con 0x7f280c06e400 2024-08-17T02:22:14.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c66add700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c600b1660 0x7f7c60136cf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c8e0 tx=0x7f7c58001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c662dc700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6006aff0 0x7f7c6013a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:14.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c65adb700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6006e140 0x7f7c601367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:14.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800ce80 con 0x7f7c600b1660 2024-08-17T02:22:14.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5800a7e0 con 0x7f7c600b1660 2024-08-17T02:22:14.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800ab60 con 0x7f7c600b1660 2024-08-17T02:22:14.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c601341e0 con 0x7f7c600b1660 2024-08-17T02:22:14.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f3290951700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3270060860 0x7f3270062d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3278000e50 tx=0x7f3278017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0daf7fe700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0d9c0609b0 0x7f0d9c062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f2808ff9700 1 -- 172.21.15.100:0/1567953067 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2804014620 con 0x7f280c06e400 2024-08-17T02:22:14.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c60134820 con 0x7f7c600b1660 2024-08-17T02:22:14.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f752e474700 1 -- 172.21.15.100:0/2757358998 shutdown_connections 2024-08-17T02:22:14.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f752e474700 1 -- 172.21.15.100:0/2757358998 wait complete. 2024-08-17T02:22:14.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f2808ff9700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27f8060860 0x7f27f8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f280bfff700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27f8060860 0x7f27f8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84015960 con 0x7efd9406dbe0 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84015ae0 con 0x7efd9406dbe0 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd940caa70 con 0x7efd9406dbe0 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd940cb030 con 0x7efd9406dbe0 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84015c90 con 0x7efd9406dbe0 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7c58012420 con 0x7f7c600b1660 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c567fc700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7c44060860 0x7f7c44062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c662dc700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7c44060860 0x7f7c44062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c600643a0 con 0x7f7c600b1660 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c662dc700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7c44060860 0x7f7c44062d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c50008b20 tx=0x7f7c50008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7c5800f260 con 0x7f7c600b1660 2024-08-17T02:22:14.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c567fc700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f7c440689f0 0x7f7c4406ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 --> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c4406b4b0 con 0x7f7c440689f0 2024-08-17T02:22:14.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c5804ca30 con 0x7f7c600b1660 2024-08-17T02:22:14.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c65adb700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f7c440689f0 0x7f7c4406ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7c65adb700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f7c440689f0 0x7f7c4406ae00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f0db4fda700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f0d9c068bc0 0x7f0d9c06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f0db4fda700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f0d9c068bc0 0x7f0d9c06afd0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 <== osd.3 v2:172.21.15.100:6808/24274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0d9c06b680 con 0x7f0d9c068bc0 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f752e474700 1 Processor -- start 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f752e474700 1 -- start start 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f752e474700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752806e450 0x7f752815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f752e474700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7528158f20 0x7f752814a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f752e474700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75281592f0 0x7f75280694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f752e474700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7528156810 con 0x7f752806e450 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f752e474700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752815a1b0 con 0x7f7528158f20 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f752e474700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752815a490 con 0x7f75281592f0 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752806e450 0x7f752815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752806e450 0x7f752815c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56886/0 (socket says 172.21.15.100:56886) 2024-08-17T02:22:14.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 -- 172.21.15.100:0/3869608189 learned_addr learned my addr 172.21.15.100:0/3869608189 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f75277fe700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7528158f20 0x7f752814a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 -- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75281592f0 msgr2=0x7f75280694c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75281592f0 0x7f75280694c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 -- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7528158f20 msgr2=0x7f752814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7528158f20 0x7f752814a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 -- 172.21.15.100:0/3869608189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c009420 con 0x7f752806e450 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f7527fff700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752806e450 0x7f752815c6f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7520002250 tx=0x7f7520001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f750ffff700 1 -- 172.21.15.100:0/3869608189 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752000ab30 con 0x7f752806e450 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f750ffff700 1 -- 172.21.15.100:0/3869608189 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752000ac90 con 0x7f752806e450 2024-08-17T02:22:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f750ffff700 1 -- 172.21.15.100:0/3869608189 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520007300 con 0x7f752806e450 2024-08-17T02:22:14.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f280bfff700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27f8060860 0x7f27f8062d10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f280c144140 tx=0x7f2800016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f2808ff9700 1 -- 172.21.15.100:0/1567953067 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2804051390 con 0x7f280c06e400 2024-08-17T02:22:14.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd940643a0 con 0x7efd9406dbe0 2024-08-17T02:22:14.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:13.996+0000 7f328bfff700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f32700689f0 0x7f327006ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f328bfff700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f32700689f0 0x7f327006ae00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 <== osd.7 v2:172.21.15.123:6824/24441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f327006b4b0 con 0x7f32700689f0 2024-08-17T02:22:14.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752815a770 con 0x7f752806e450 2024-08-17T02:22:14.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75281564a0 con 0x7f752806e450 2024-08-17T02:22:14.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 <== osd.0 v2:172.21.15.100:6800/24271 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f7c4406b4b0 con 0x7f7c440689f0 2024-08-17T02:22:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f2812f1e700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f280c171530 0x7f280c171900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 --> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f280c171e40 con 0x7f280c171530 2024-08-17T02:22:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f28114bb700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f280c171530 0x7f280c171900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f28114bb700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f280c171530 0x7f280c171900 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f750ffff700 1 -- 172.21.15.100:0/3869608189 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f75200076f0 con 0x7f752806e450 2024-08-17T02:22:14.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f750ffff700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7514060860 0x7f7514062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f2808ff9700 1 -- 172.21.15.100:0/1567953067 <== osd.5 v2:172.21.15.123:6800/24438 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f280c171e40 con 0x7f280c171530 2024-08-17T02:22:14.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f750ffff700 1 -- 172.21.15.100:0/3869608189 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f752007d0b0 con 0x7f752806e450 2024-08-17T02:22:14.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7f75277fe700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7514060860 0x7f7514062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efd8401b020 con 0x7efd9406dbe0 2024-08-17T02:22:14.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7efd90ff9700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd80060930 0x7efd80062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd84084fd0 con 0x7efd9406dbe0 2024-08-17T02:22:14.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7efd90ff9700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7efd80068b40 0x7efd8006af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7efd93fff700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd80060930 0x7efd80062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 --> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd8006b600 con 0x7efd80068b40 2024-08-17T02:22:14.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.000+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd840853f0 con 0x7efd9406dbe0 2024-08-17T02:22:14.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7efd991a1700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7efd80068b40 0x7efd8006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7efd991a1700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7efd80068b40 0x7efd8006af50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7efd93fff700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd80060930 0x7efd80062de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0097c0 tx=0x7efd8c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.004+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 <== osd.6 v2:172.21.15.123:6816/24440 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7efd8006b600 con 0x7efd80068b40 2024-08-17T02:22:14.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.008+0000 7f0daf7fe700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0d9c0609b0 0x7f0d9c062e60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0db00ca700 tx=0x7f0da4007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.008+0000 7f75277fe700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7514060860 0x7f7514062d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098e0 tx=0x7f751c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.012+0000 7f752e474700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f752814a980 0x7f7528069a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.012+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 --> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75280643a0 con 0x7f752814a980 2024-08-17T02:22:14.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.012+0000 7f752ca11700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f752814a980 0x7f7528069a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.012+0000 7f752ca11700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f752814a980 0x7f7528069a00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.012+0000 7f750ffff700 1 -- 172.21.15.100:0/3869608189 <== osd.4 v2:172.21.15.123:6803/24439 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f75280643a0 con 0x7f752814a980 2024-08-17T02:22:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.016+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 --> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f862c06f5f0 con 0x7f862c06a3d0 2024-08-17T02:22:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.016+0000 7f86137fe700 1 -- 172.21.15.100:0/1528768808 <== osd.1 v2:172.21.15.100:6824/24273 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f862c06f5f0 con 0x7f862c06a3d0 2024-08-17T02:22:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f3292bb5700 1 -- 172.21.15.100:0/3131254803 --> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f328403a580 con 0x7f32700689f0 2024-08-17T02:22:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f862c06a3d0 msgr2=0x7f862c0c7ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f862c06a3d0 0x7f862c0c7ce0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f86180608e0 msgr2=0x7f8618062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f86180608e0 0x7f8618062d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8620006fd0 tx=0x7f8620008040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f862c13b630 msgr2=0x7f862c163930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 --2- 172.21.15.100:0/1528768808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f862c13b630 0x7f862c163930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f861c013000 tx=0x7f861c017000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f3288ff9700 1 -- 172.21.15.100:0/3131254803 <== osd.7 v2:172.21.15.123:6824/24441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f328403a580 con 0x7f32700689f0 2024-08-17T02:22:14.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f862bfff700 1 -- 172.21.15.100:0/1528768808 reap_dead start 2024-08-17T02:22:14.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 shutdown_connections 2024-08-17T02:22:14.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 >> 172.21.15.100:0/1528768808 conn(0x7f862c0678d0 msgr2=0x7f862c06dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 shutdown_connections 2024-08-17T02:22:14.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.020+0000 7f8631efd700 1 -- 172.21.15.100:0/1528768808 wait complete. 2024-08-17T02:22:14.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 --> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f489c0643a0 con 0x7f4888068a70 2024-08-17T02:22:14.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 -- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f32700689f0 msgr2=0x7f327006ae00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f32700689f0 0x7f327006ae00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 -- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3270060860 msgr2=0x7f3270062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3270060860 0x7f3270062d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3278000e50 tx=0x7f3278017000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 -- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32840e3b70 msgr2=0x7f32840f3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 --2- 172.21.15.100:0/3131254803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32840e3b70 0x7f32840f3ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f328c02b300 tx=0x7f328c065310 comp rx=0 tx=0).stop 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f3291152700 1 -- 172.21.15.100:0/3131254803 reap_dead start 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 -- 172.21.15.100:0/3131254803 shutdown_connections 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 -- 172.21.15.100:0/3131254803 >> 172.21.15.100:0/3131254803 conn(0x7f3284057900 msgr2=0x7f32840562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f4883fff700 1 -- 172.21.15.100:0/4039682757 <== osd.2 v2:172.21.15.100:6809/24272 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f489c0643a0 con 0x7f4888068a70 2024-08-17T02:22:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 -- 172.21.15.100:0/3131254803 shutdown_connections 2024-08-17T02:22:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f32767fc700 1 -- 172.21.15.100:0/3131254803 wait complete. 2024-08-17T02:22:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7f4888068a70 msgr2=0x7f488806ae80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7f4888068a70 0x7f488806ae80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4888060860 msgr2=0x7f4888062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4888060860 0x7f4888062d10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f489c06e0f0 tx=0x7f488c008040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489c0b1950 msgr2=0x7f489c158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 --2- 172.21.15.100:0/4039682757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489c0b1950 0x7f489c158290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4894017000 tx=0x7f489401a000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a0fe9700 1 -- 172.21.15.100:0/4039682757 reap_dead start 2024-08-17T02:22:14.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 shutdown_connections 2024-08-17T02:22:14.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 >> 172.21.15.100:0/4039682757 conn(0x7f489c0678d0 msgr2=0x7f489c06a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 shutdown_connections 2024-08-17T02:22:14.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f48a2a4c700 1 -- 172.21.15.100:0/4039682757 wait complete. 2024-08-17T02:22:14.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f0db6a3d700 1 -- 172.21.15.100:0/2719135166 --> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0d90002d30 con 0x7f0d9c068bc0 2024-08-17T02:22:14.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f0d97fff700 1 -- 172.21.15.100:0/2719135166 <== osd.3 v2:172.21.15.100:6808/24274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0d90002d30 con 0x7f0d9c068bc0 2024-08-17T02:22:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f0d95ffb700 1 -- 172.21.15.100:0/2719135166 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f0d9c068bc0 msgr2=0x7f0d9c06afd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.024+0000 7f0d95ffb700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f0d9c068bc0 0x7f0d9c06afd0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.028+0000 7f0d95ffb700 1 -- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0d9c0609b0 msgr2=0x7f0d9c062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.028+0000 7f0d95ffb700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0d9c0609b0 0x7f0d9c062e60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0db00ca700 tx=0x7f0da4007ec0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.028+0000 7f0d95ffb700 1 -- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db00b1950 msgr2=0x7f0db016a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.028+0000 7f0d95ffb700 1 --2- 172.21.15.100:0/2719135166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db00b1950 0x7f0db016a160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0da800a8d0 tx=0x7f0da8019000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.028+0000 7f0db4fda700 1 -- 172.21.15.100:0/2719135166 reap_dead start 2024-08-17T02:22:14.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.028+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 --> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f280c0643a0 con 0x7f280c171530 2024-08-17T02:22:14.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.028+0000 7f2808ff9700 1 -- 172.21.15.100:0/1567953067 <== osd.5 v2:172.21.15.123:6800/24438 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f280c0643a0 con 0x7f280c171530 2024-08-17T02:22:14.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f0d95ffb700 1 -- 172.21.15.100:0/2719135166 shutdown_connections 2024-08-17T02:22:14.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f0d95ffb700 1 -- 172.21.15.100:0/2719135166 >> 172.21.15.100:0/2719135166 conn(0x7f0db00678d0 msgr2=0x7f0db0154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f280c171530 msgr2=0x7f280c171900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f280c171530 0x7f280c171900 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 --> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd940643a0 con 0x7efd80068b40 2024-08-17T02:22:14.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27f8060860 msgr2=0x7f27f8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27f8060860 0x7f27f8062d10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f280c144140 tx=0x7f2800016000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e400 msgr2=0x7f280c143190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 --2- 172.21.15.100:0/1567953067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c06e400 0x7f280c143190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f280400bb00 tx=0x7f2804009780 comp rx=0 tx=0).stop 2024-08-17T02:22:14.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f28114bb700 1 -- 172.21.15.100:0/1567953067 reap_dead start 2024-08-17T02:22:14.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 shutdown_connections 2024-08-17T02:22:14.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 >> 172.21.15.100:0/1567953067 conn(0x7f280c067890 msgr2=0x7f280c068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f0d95ffb700 1 -- 172.21.15.100:0/2719135166 shutdown_connections 2024-08-17T02:22:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f0d95ffb700 1 -- 172.21.15.100:0/2719135166 wait complete. 2024-08-17T02:22:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 shutdown_connections 2024-08-17T02:22:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7efd90ff9700 1 -- 172.21.15.100:0/1330963715 <== osd.6 v2:172.21.15.123:6816/24440 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efd940643a0 con 0x7efd80068b40 2024-08-17T02:22:14.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f2812f1e700 1 -- 172.21.15.100:0/1567953067 wait complete. 2024-08-17T02:22:14.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7efd80068b40 msgr2=0x7efd8006af50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7efd999a2700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7efd80068b40 0x7efd8006af50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 --> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c60134370 con 0x7f7c440689f0 2024-08-17T02:22:14.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c567fc700 1 -- 172.21.15.100:0/3672744040 <== osd.0 v2:172.21.15.100:6800/24271 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7c60134370 con 0x7f7c440689f0 2024-08-17T02:22:14.090 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:14.091 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-17T02:22:14.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f7c440689f0 msgr2=0x7f7c4406ae00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c672de700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f7c440689f0 0x7f7c4406ae00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7c44060860 msgr2=0x7f7c44062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c672de700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7c44060860 0x7f7c44062d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c50008b20 tx=0x7f7c50008040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c600b1660 msgr2=0x7f7c60136cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c672de700 1 --2- 172.21.15.100:0/3672744040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c600b1660 0x7f7c60136cf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c8e0 tx=0x7f7c58001710 comp rx=0 tx=0).stop 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.036+0000 7f7c66add700 1 -- 172.21.15.100:0/3672744040 reap_dead start 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.040+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 shutdown_connections 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.040+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 >> 172.21.15.100:0/3672744040 conn(0x7f7c600678d0 msgr2=0x7f7c600adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.040+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 shutdown_connections 2024-08-17T02:22:14.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.040+0000 7f7c672de700 1 -- 172.21.15.100:0/3672744040 wait complete. 2024-08-17T02:22:14.094 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:14.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-17T02:22:14.095 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:14.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-17T02:22:14.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd80060930 msgr2=0x7efd80062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd999a2700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd80060930 0x7efd80062de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0097c0 tx=0x7efd8c017000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9406dbe0 msgr2=0x7efd9415b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd999a2700 1 --2- 172.21.15.100:0/1330963715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9406dbe0 0x7efd9415b910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd84007fb0 tx=0x7efd84007b70 comp rx=0 tx=0).stop 2024-08-17T02:22:14.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd991a1700 1 -- 172.21.15.100:0/1330963715 reap_dead start 2024-08-17T02:22:14.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 --> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7528159f90 con 0x7f752814a980 2024-08-17T02:22:14.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 shutdown_connections 2024-08-17T02:22:14.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 >> 172.21.15.100:0/1330963715 conn(0x7efd94066e00 msgr2=0x7efd940b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 shutdown_connections 2024-08-17T02:22:14.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7efd999a2700 1 -- 172.21.15.100:0/1330963715 wait complete. 2024-08-17T02:22:14.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f750ffff700 1 -- 172.21.15.100:0/3869608189 <== osd.4 v2:172.21.15.123:6803/24439 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7528159f90 con 0x7f752814a980 2024-08-17T02:22:14.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f752814a980 msgr2=0x7f7528069a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f752814a980 0x7f7528069a00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7514060860 msgr2=0x7f7514062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7514060860 0x7f7514062d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098e0 tx=0x7f751c017000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752806e450 msgr2=0x7f752815c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 --2- 172.21.15.100:0/3869608189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752806e450 0x7f752815c6f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7520002250 tx=0x7f7520001bb0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752ca11700 1 -- 172.21.15.100:0/3869608189 reap_dead start 2024-08-17T02:22:14.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 shutdown_connections 2024-08-17T02:22:14.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 >> 172.21.15.100:0/3869608189 conn(0x7f75280678d0 msgr2=0x7f7528154700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 shutdown_connections 2024-08-17T02:22:14.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.044+0000 7f752e474700 1 -- 172.21.15.100:0/3869608189 wait complete. 2024-08-17T02:22:14.100 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:14.101 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-17T02:22:14.102 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:14.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-17T02:22:14.103 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:14.103 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-17T02:22:14.104 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:14.104 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-17T02:22:14.105 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:14.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-17T02:22:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f4e0b59e700 1 Processor -- start 2024-08-17T02:22:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f4e0b59e700 1 -- start start 2024-08-17T02:22:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7fa18e489700 1 Processor -- start 2024-08-17T02:22:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f559c9d1700 1 Processor -- start 2024-08-17T02:22:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7fa18e489700 1 -- start start 2024-08-17T02:22:14.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f559c9d1700 1 -- start start 2024-08-17T02:22:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559c9d1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f559814f850 0x7f55980691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559c9d1700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f559814fc20 0x7f5598156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559c9d1700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5598069a40 con 0x7f55980ae5d0 2024-08-17T02:22:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559c9d1700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5598069d20 con 0x7f559814fc20 2024-08-17T02:22:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559c9d1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559806a000 con 0x7f559814f850 2024-08-17T02:22:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18e489700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa1880d0010 0x7fa1880d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18e489700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa18815cea0 0x7fa18815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18e489700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1880cf190 con 0x7fa188159040 2024-08-17T02:22:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18e489700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1880d09a0 con 0x7fa1880d0010 2024-08-17T02:22:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18e489700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1880ce5e0 con 0x7fa18815cea0 2024-08-17T02:22:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f559814f850 0x7f55980691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f559814f850 0x7f55980691f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:37622/0 (socket says 172.21.15.100:37622) 2024-08-17T02:22:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559659c700 1 -- 172.21.15.100:0/3453079306 learned_addr learned my addr 172.21.15.100:0/3453079306 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f5595d9b700 1 --2- 172.21.15.100:0/3453079306 >> v2:172.21.15.100:3300/0 conn(0x7f559814fc20 0x7f5598156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18cc86700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa1880d0010 0x7fa1880d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18d487700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa188159040 0x7fa1880cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50868/0 (socket says 172.21.15.100:50868) 2024-08-17T02:22:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18d487700 1 -- 172.21.15.100:0/2858810108 learned_addr learned my addr 172.21.15.100:0/2858810108 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f4e0b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0ca0d0 0x7f4e0c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f4e0b59e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c0cae00 con 0x7f4e0c15d230 2024-08-17T02:22:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f4e0b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c0acf90 con 0x7f4e0c0cb080 2024-08-17T02:22:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f4e0b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0cab20 con 0x7f4e0c0ca0d0 2024-08-17T02:22:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f4e0ad9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4e0c0cb080 0x7f4e0c0ad420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:34036/0 (socket says 172.21.15.100:34036) 2024-08-17T02:22:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7f4e0ad9d700 1 -- 172.21.15.100:0/1626401741 learned_addr learned my addr 172.21.15.100:0/1626401741 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f4e09d9b700 1 --2- 172.21.15.100:0/1626401741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0ca0d0 0x7f4e0c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559659c700 1 -- 172.21.15.100:0/3453079306 >> v2:172.21.15.100:3300/0 conn(0x7f559814fc20 msgr2=0x7f5598156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559659c700 1 --2- 172.21.15.100:0/3453079306 >> v2:172.21.15.100:3300/0 conn(0x7f559814fc20 0x7f5598156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559659c700 1 -- 172.21.15.100:0/3453079306 >> v1:172.21.15.100:6789/0 conn(0x7f55980ae5d0 legacy=0x7f55980aa440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:22:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f4e09d9b700 1 -- 172.21.15.100:0/1626401741 >> v1:172.21.15.123:6789/0 conn(0x7f4e0c0cb080 legacy=0x7f4e0c0ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f4e09d9b700 1 -- 172.21.15.100:0/1626401741 >> v1:172.21.15.100:6789/0 conn(0x7f4e0c15d230 legacy=0x7f4e0c15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559659c700 1 -- 172.21.15.100:0/3453079306 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598155ce0 con 0x7f559814f850 2024-08-17T02:22:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f4e09d9b700 1 -- 172.21.15.100:0/1626401741 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c0ac940 con 0x7f4e0c0ca0d0 2024-08-17T02:22:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f559659c700 1 --2- 172.21.15.100:0/3453079306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559814f850 0x7f55980691f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f558c0099b0 tx=0x7f558c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=372eececf6cf861 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f4e09d9b700 1 --2- 172.21.15.100:0/1626401741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0ca0d0 0x7f4e0c0ca520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009910 tx=0x7f4e000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed152d845e98d01 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7fa35ccab700 1 Processor -- start 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.284+0000 7fa35ccab700 1 -- start start 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35ccab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa358159340 0x7fa358159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35ccab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3580b4330 0x7fa3580b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35ccab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3580b1cf0 con 0x7fa358159c50 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35ccab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3580b60d0 con 0x7fa358159340 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35ccab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580b10e0 con 0x7fa3580b4330 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa358159340 0x7fa358159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa358159340 0x7fa358159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:37616/0 (socket says 172.21.15.100:37616) 2024-08-17T02:22:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35659c700 1 -- 172.21.15.100:0/271721392 learned_addr learned my addr 172.21.15.100:0/271721392 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35659c700 1 -- 172.21.15.100:0/271721392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3580b4330 msgr2=0x7fa3580b04e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:22:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35659c700 1 --2- 172.21.15.100:0/271721392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3580b4330 0x7fa3580b04e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35659c700 1 -- 172.21.15.100:0/271721392 >> v1:172.21.15.123:6789/0 conn(0x7fa358159c50 legacy=0x7fa3580b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35659c700 1 -- 172.21.15.100:0/271721392 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580b13c0 con 0x7fa358159340 2024-08-17T02:22:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa35659c700 1 --2- 172.21.15.100:0/271721392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa358159340 0x7fa358159710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa350009f20 tx=0x7fa350009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b3d30412f9cd03 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18cc86700 1 -- 172.21.15.100:0/2858810108 >> v2:172.21.15.123:3300/0 conn(0x7fa18815cea0 msgr2=0x7fa18815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18dc88700 1 --2- 172.21.15.100:0/2858810108 >> v2:172.21.15.123:3300/0 conn(0x7fa18815cea0 0x7fa18815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18cc86700 1 --2- 172.21.15.100:0/2858810108 >> v2:172.21.15.123:3300/0 conn(0x7fa18815cea0 0x7fa18815d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18cc86700 1 -- 172.21.15.100:0/2858810108 >> v1:172.21.15.100:6789/0 conn(0x7fa188159040 legacy=0x7fa1880cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18cc86700 1 -- 172.21.15.100:0/2858810108 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1880ce860 con 0x7fa1880d0010 2024-08-17T02:22:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7fa18cc86700 1 --2- 172.21.15.100:0/2858810108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1880d0010 0x7fa1880d0460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa1880b6050 tx=0x7fa1780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66a65352c8e6a45 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f15d22ab700 1 Processor -- start 2024-08-17T02:22:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f487421c700 1 Processor -- start 2024-08-17T02:22:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f15d22ab700 1 -- start start 2024-08-17T02:22:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.288+0000 7f487421c700 1 -- start start 2024-08-17T02:22:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf2abf700 1 Processor -- start 2024-08-17T02:22:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a86e64700 1 Processor -- start 2024-08-17T02:22:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf2abf700 1 -- start start 2024-08-17T02:22:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0959a700 1 -- 172.21.15.100:0/1626401741 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000b860 con 0x7f4e0c0ca0d0 2024-08-17T02:22:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0959a700 1 -- 172.21.15.100:0/1626401741 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0000b9c0 con 0x7f4e0c0ca0d0 2024-08-17T02:22:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0959a700 1 -- 172.21.15.100:0/1626401741 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000bd60 con 0x7f4e0c0ca0d0 2024-08-17T02:22:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa17ffff700 1 -- 172.21.15.100:0/2858810108 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800b6b0 con 0x7fa1880d0010 2024-08-17T02:22:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa17ffff700 1 -- 172.21.15.100:0/2858810108 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17800b810 con 0x7fa1880d0010 2024-08-17T02:22:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa17ffff700 1 -- 172.21.15.100:0/2858810108 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800bbb0 con 0x7fa1880d0010 2024-08-17T02:22:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15d22ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa18e489700 1 -- 172.21.15.100:0/2858810108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1880d0010 msgr2=0x7fa1880d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa18e489700 1 --2- 172.21.15.100:0/2858810108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1880d0010 0x7fa1880d0460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa1880b6050 tx=0x7fa1780091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15d22ab700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15cc0afb90 0x7f15cc0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15d22ab700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc0ad280 con 0x7f15cc13f9b0 2024-08-17T02:22:14.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15d22ab700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc06ea80 con 0x7f15cc0afb90 2024-08-17T02:22:14.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15d22ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0adc50 con 0x7f15cc13f5e0 2024-08-17T02:22:14.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f487421c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 0x7f486c06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f487421c700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486c0c87d0 con 0x7f486c155760 2024-08-17T02:22:14.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f487421c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486c0ac7d0 con 0x7f486c0aa580 2024-08-17T02:22:14.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f487421c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486c06e9c0 con 0x7f486c155b30 2024-08-17T02:22:14.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf2abf700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbbec155760 0x7fbbec0abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf2abf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155b30 0x7fbbec0a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf2abf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbec0aa580 0x7fbbec0a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf2abf700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbec0c87d0 con 0x7fbbec155760 2024-08-17T02:22:14.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf2abf700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbec0ac7d0 con 0x7fbbec0aa580 2024-08-17T02:22:14.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf2abf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec0acab0 con 0x7fbbec155b30 2024-08-17T02:22:14.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35559a700 1 -- 172.21.15.100:0/271721392 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35000e070 con 0x7fa358159340 2024-08-17T02:22:14.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35559a700 1 -- 172.21.15.100:0/271721392 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35000ba80 con 0x7fa358159340 2024-08-17T02:22:14.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35559a700 1 -- 172.21.15.100:0/271721392 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3500092e0 con 0x7fa358159340 2024-08-17T02:22:14.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a86e64700 1 -- start start 2024-08-17T02:22:14.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4871fb8700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f486c155760 0x7f486c0abf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50884/0 (socket says 172.21.15.100:50884) 2024-08-17T02:22:14.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4871fb8700 1 -- 172.21.15.100:0/3175926579 learned_addr learned my addr 172.21.15.100:0/3175926579 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:14.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15cb7fe700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f15cc13f9b0 0x7f15cc06df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50886/0 (socket says 172.21.15.100:50886) 2024-08-17T02:22:14.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15cb7fe700 1 -- 172.21.15.100:0/3982185649 learned_addr learned my addr 172.21.15.100:0/3982185649 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:14.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa18e489700 1 -- 172.21.15.100:0/2858810108 shutdown_connections 2024-08-17T02:22:14.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa18e489700 1 --2- 172.21.15.100:0/2858810108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1880d0010 0x7fa1880d0460 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa18e489700 1 --2- 172.21.15.100:0/2858810108 >> v2:172.21.15.123:3300/0 conn(0x7fa18815cea0 0x7fa18815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa18e489700 1 -- 172.21.15.100:0/2858810108 >> 172.21.15.100:0/2858810108 conn(0x7fa188068250 msgr2=0x7fa188068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15cbfff700 1 --2- 172.21.15.100:0/3982185649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561526608 0 0) 0x7f486c0c87d0 con 0x7f486c155760 2024-08-17T02:22:14.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15caffd700 1 -- 172.21.15.100:0/3982185649 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814114272 0 0) 0x7f15cc0ad280 con 0x7f15cc13f9b0 2024-08-17T02:22:14.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15d0848700 1 --2- 172.21.15.100:0/3982185649 >> v2:172.21.15.100:3300/0 conn(0x7f15cc0afb90 0x7f15cc0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15caffd700 1 -- 172.21.15.100:0/3982185649 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15b4003570 con 0x7f15cc13f9b0 2024-08-17T02:22:14.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15cbfff700 1 -- 172.21.15.100:0/3982185649 >> v2:172.21.15.100:3300/0 conn(0x7f15cc0afb90 msgr2=0x7f15cc0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15cbfff700 1 --2- 172.21.15.100:0/3982185649 >> v2:172.21.15.100:3300/0 conn(0x7f15cc0afb90 0x7f15cc0afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15cbfff700 1 -- 172.21.15.100:0/3982185649 >> v1:172.21.15.100:6789/0 conn(0x7f15cc13f9b0 legacy=0x7f15cc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15cbfff700 1 -- 172.21.15.100:0/3982185649 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc0b0530 con 0x7f15cc13f5e0 2024-08-17T02:22:14.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15d0848700 1 --2- 172.21.15.100:0/3982185649 >> v2:172.21.15.100:3300/0 conn(0x7f15cc0afb90 0x7f15cc0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:14.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf085b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbbec155760 0x7fbbec0abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4858003570 con 0x7f486c155760 2024-08-17T02:22:14.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0b59e700 1 -- 172.21.15.100:0/1626401741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0ca0d0 msgr2=0x7f4e0c0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0b59e700 1 --2- 172.21.15.100:0/1626401741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0ca0d0 0x7f4e0c0ca520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009910 tx=0x7f4e000091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f48717b7700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 0x7f486c06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559559a700 1 -- 172.21.15.100:0/3453079306 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c009ea0 con 0x7f559814f850 2024-08-17T02:22:14.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559559a700 1 -- 172.21.15.100:0/3453079306 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c00b7e0 con 0x7f559814f850 2024-08-17T02:22:14.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559559a700 1 -- 172.21.15.100:0/3453079306 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c00bb80 con 0x7f559814f850 2024-08-17T02:22:14.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35ccab700 1 -- 172.21.15.100:0/271721392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa358159340 msgr2=0x7fa358159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35ccab700 1 --2- 172.21.15.100:0/271721392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa358159340 0x7fa358159710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa350009f20 tx=0x7fa350009840 comp rx=0 tx=0).stop 2024-08-17T02:22:14.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559c9d1700 1 -- 172.21.15.100:0/3453079306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559814f850 msgr2=0x7f55980691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559c9d1700 1 --2- 172.21.15.100:0/3453079306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559814f850 0x7f55980691f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f558c0099b0 tx=0x7f558c009250 comp rx=0 tx=0).stop 2024-08-17T02:22:14.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559c9d1700 1 -- 172.21.15.100:0/3453079306 shutdown_connections 2024-08-17T02:22:14.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559c9d1700 1 --2- 172.21.15.100:0/3453079306 >> v2:172.21.15.100:3300/0 conn(0x7f559814fc20 0x7f5598156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559c9d1700 1 --2- 172.21.15.100:0/3453079306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559814f850 0x7f55980691f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf085b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbbec155760 0x7fbbec0abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52238/0 (socket says 172.21.15.100:52238) 2024-08-17T02:22:14.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf085b700 1 -- 172.21.15.100:0/336342083 learned_addr learned my addr 172.21.15.100:0/336342083 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559c9d1700 1 -- 172.21.15.100:0/3453079306 >> 172.21.15.100:0/3453079306 conn(0x7f5598068300 msgr2=0x7f55980a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15cbfff700 1 --2- 172.21.15.100:0/3982185649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc0ad710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0098e0 tx=0x7f15bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad65e072414ff601 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35ccab700 1 -- 172.21.15.100:0/271721392 shutdown_connections 2024-08-17T02:22:14.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35ccab700 1 --2- 172.21.15.100:0/271721392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3580b4330 0x7fa3580b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35ccab700 1 --2- 172.21.15.100:0/271721392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa358159340 0x7fa358159710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35ccab700 1 -- 172.21.15.100:0/271721392 >> 172.21.15.100:0/271721392 conn(0x7fa3580994f0 msgr2=0x7fa3580675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559c9d1700 1 -- 172.21.15.100:0/3453079306 shutdown_connections 2024-08-17T02:22:14.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f559c9d1700 1 -- 172.21.15.100:0/3453079306 wait complete. 2024-08-17T02:22:14.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0b59e700 1 -- 172.21.15.100:0/1626401741 shutdown_connections 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822561735 0 0) 0x7f4858003570 con 0x7f486c155760 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 msgr2=0x7f486c06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 0x7f486c06eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 >> v1:172.21.15.123:6789/0 conn(0x7f486c0aa580 legacy=0x7f486c06e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c0acab0 con 0x7f486c155760 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f4868001470 con 0x7f486c155760 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 >> v1:172.21.15.100:6789/0 conn(0x7f486c155760 legacy=0x7f486c0abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4858003b10 0x7f4858003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858004810 0x7f4858004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 0x7f4858014270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155b30 0x7fbbec0a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0b59e700 1 --2- 172.21.15.100:0/1626401741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0ca0d0 0x7f4e0c0ca520 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0b59e700 1 -- 172.21.15.100:0/1626401741 >> 172.21.15.100:0/1626401741 conn(0x7f4e0c0cd860 msgr2=0x7f4e0c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4871fb8700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4858003b10 0x7f4858003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf085b700 1 -- 172.21.15.100:0/336342083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155b30 msgr2=0x7fbbec0a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf085b700 1 --2- 172.21.15.100:0/336342083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155b30 0x7fbbec0a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a86e64700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8006e140 0x7f0a8006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a86e64700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0a8006aff0 0x7f0a801556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf085b700 1 -- 172.21.15.100:0/336342083 >> v2:172.21.15.123:3300/0 conn(0x7fbbec0aa580 msgr2=0x7fbbec0a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a86e64700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0a800b1660 0x7f0a80158d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a86e64700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a80156940 con 0x7f0a8006aff0 2024-08-17T02:22:14.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a86e64700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a80155d30 con 0x7f0a800b1660 2024-08-17T02:22:14.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a86e64700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80156010 con 0x7f0a8006e140 2024-08-17T02:22:14.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f48727b9700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 0x7f4858014270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c06e9c0 con 0x7f4858003b10 2024-08-17T02:22:14.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4858005540 con 0x7f4858004810 2024-08-17T02:22:14.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48580147b0 con 0x7f486c155b30 2024-08-17T02:22:14.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35ccab700 1 -- 172.21.15.100:0/271721392 shutdown_connections 2024-08-17T02:22:14.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa35ccab700 1 -- 172.21.15.100:0/271721392 wait complete. 2024-08-17T02:22:14.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85661700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0a8006aff0 0x7f0a801556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0b59e700 1 -- 172.21.15.100:0/1626401741 shutdown_connections 2024-08-17T02:22:14.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fa18e489700 1 -- 172.21.15.100:0/2858810108 shutdown_connections 2024-08-17T02:22:14.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85e62700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8006e140 0x7f0a8006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85661700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0a8006aff0 0x7f0a801556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52266/0 (socket says 172.21.15.100:52266) 2024-08-17T02:22:14.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85661700 1 -- 172.21.15.100:0/1509785302 learned_addr learned my addr 172.21.15.100:0/1509785302 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf105c700 1 --2- 172.21.15.100:0/336342083 >> v2:172.21.15.123:3300/0 conn(0x7fbbec0aa580 0x7fbbec0a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf085b700 1 --2- 172.21.15.100:0/336342083 >> v2:172.21.15.123:3300/0 conn(0x7fbbec0aa580 0x7fbbec0a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf085b700 1 -- 172.21.15.100:0/336342083 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec0acd90 con 0x7fbbec155760 2024-08-17T02:22:14.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7fbbf105c700 1 --2- 172.21.15.100:0/336342083 >> v2:172.21.15.123:3300/0 conn(0x7fbbec0aa580 0x7fbbec0a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:14.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4e0b59e700 1 -- 172.21.15.100:0/1626401741 wait complete. 2024-08-17T02:22:14.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85661700 1 -- 172.21.15.100:0/1509785302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8006e140 msgr2=0x7f0a8006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85661700 1 --2- 172.21.15.100:0/1509785302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8006e140 0x7f0a8006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85661700 1 -- 172.21.15.100:0/1509785302 >> v2:172.21.15.123:3300/0 conn(0x7f0a800b1660 msgr2=0x7f0a80158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85661700 1 --2- 172.21.15.100:0/1509785302 >> v2:172.21.15.123:3300/0 conn(0x7f0a800b1660 0x7f0a80158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f0a85661700 1 -- 172.21.15.100:0/1509785302 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a801562f0 con 0x7f0a8006aff0 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4871fb8700 1 -- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 msgr2=0x7f4858014270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4871fb8700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 0x7f4858014270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4871fb8700 1 -- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858004810 msgr2=0x7f4858004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4871fb8700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858004810 0x7f4858004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f4871fb8700 1 -- 172.21.15.100:0/3175926579 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858003570 con 0x7f4858003b10 2024-08-17T02:22:14.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.292+0000 7f15caffd700 1 -- 172.21.15.100:0/3982185649 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00b640 con 0x7f15cc13f5e0 2024-08-17T02:22:14.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f15caffd700 1 -- 172.21.15.100:0/3982185649 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc00b7a0 con 0x7f15cc13f5e0 2024-08-17T02:22:14.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f15caffd700 1 -- 172.21.15.100:0/3982185649 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00bb60 con 0x7f15cc13f5e0 2024-08-17T02:22:14.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf085b700 1 --2- 172.21.15.100:0/336342083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec155760 0x7fbbec0abf80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0704d0 tx=0x7fbbe4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf0b40d1fb80740 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4871fb8700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4858003b10 0x7f4858003f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f486800c2d0 tx=0x7f4868001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf05d9fa563f96c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 Processor -- start 2024-08-17T02:22:14.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f48727b9700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c155b30 0x7f4858014270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:14.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486800cba0 con 0x7f4858003b10 2024-08-17T02:22:14.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486800cd00 con 0x7f4858003b10 2024-08-17T02:22:14.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4870fb6700 1 -- 172.21.15.100:0/3175926579 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868005b70 con 0x7f4858003b10 2024-08-17T02:22:14.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 Processor -- start 2024-08-17T02:22:14.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 -- start start 2024-08-17T02:22:14.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f15d22ab700 1 -- 172.21.15.100:0/3982185649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 msgr2=0x7f15cc0ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f15d22ab700 1 --2- 172.21.15.100:0/3982185649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc0ad710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0098e0 tx=0x7f15bc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f0a85661700 1 --2- 172.21.15.100:0/1509785302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006aff0 0x7f0a801556f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a800b13e0 tx=0x7f0a7c00a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3277716fb5bf69b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa18e489700 1 -- 172.21.15.100:0/2858810108 wait complete. 2024-08-17T02:22:14.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 -- start start 2024-08-17T02:22:14.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f487421c700 1 -- 172.21.15.100:0/3175926579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4858003b10 msgr2=0x7f4858003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f487421c700 1 --2- 172.21.15.100:0/3175926579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4858003b10 0x7f4858003f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f486800c2d0 tx=0x7f4868001da0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55980ae5d0 0x7f5598151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f559814f850 0x7f5598171050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f48727b9700 1 -- 172.21.15.100:0/3175926579 reap_dead start 2024-08-17T02:22:14.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559814fc20 0x7f55981492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55981523c0 con 0x7f559814fc20 2024-08-17T02:22:14.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5598098140 con 0x7f55980ae5d0 2024-08-17T02:22:14.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5598097530 con 0x7f559814f850 2024-08-17T02:22:14.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0b59e700 1 Processor -- start 2024-08-17T02:22:14.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0b59e700 1 -- start start 2024-08-17T02:22:14.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbeb7fe700 1 -- 172.21.15.100:0/336342083 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400e070 con 0x7fbbec155760 2024-08-17T02:22:14.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559814fc20 0x7f55981492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559814fc20 0x7f55981492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52268/0 (socket says 172.21.15.100:52268) 2024-08-17T02:22:14.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 -- 172.21.15.100:0/2188454451 learned_addr learned my addr 172.21.15.100:0/2188454451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5595d9b700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f559814f850 0x7f5598171050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbeb7fe700 1 -- 172.21.15.100:0/336342083 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe400ba50 con 0x7fbbec155760 2024-08-17T02:22:14.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbeb7fe700 1 -- 172.21.15.100:0/336342083 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400e070 con 0x7fbbec155760 2024-08-17T02:22:14.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559659c700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55980ae5d0 0x7f5598151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3580b4330 0x7fa3580d0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa358159340 0x7fa358169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa358159c50 0x7fa3580c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580affa0 con 0x7fa3580b4330 2024-08-17T02:22:14.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580ac3b0 con 0x7fa358159c50 2024-08-17T02:22:14.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580ac690 con 0x7fa358159340 2024-08-17T02:22:14.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3580b4330 0x7fa3580d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3580b4330 0x7fa3580d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52280/0 (socket says 172.21.15.100:52280) 2024-08-17T02:22:14.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 -- 172.21.15.100:0/271815029 learned_addr learned my addr 172.21.15.100:0/271815029 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f487421c700 1 -- 172.21.15.100:0/3175926579 shutdown_connections 2024-08-17T02:22:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f487421c700 1 -- 172.21.15.100:0/3175926579 >> 172.21.15.100:0/3175926579 conn(0x7f486c067810 msgr2=0x7f486c067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 -- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f559814f850 msgr2=0x7f5598171050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f559814f850 0x7f5598171050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 -- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55980ae5d0 msgr2=0x7f5598151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55980ae5d0 0x7f5598151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 -- 172.21.15.100:0/2188454451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c0094f0 con 0x7f559814fc20 2024-08-17T02:22:14.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f5596d9d700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559814fc20 0x7f55981492b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f559000c930 tx=0x7f55900026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa355d9b700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa358159340 0x7fa358169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 -- 172.21.15.100:0/336342083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec155760 msgr2=0x7fbbec0abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/336342083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec155760 0x7fbbec0abf80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0704d0 tx=0x7fbbe4009840 comp rx=0 tx=0).stop 2024-08-17T02:22:14.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e0c0ca0d0 0x7f4e0c162010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0cb080 0x7f4e0c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0b59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e0c15d230 0x7f4e0c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 -- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa358159340 msgr2=0x7fa358169740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa358159340 0x7fa358169740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 -- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa358159c50 msgr2=0x7fa3580c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa358159c50 0x7fa3580c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 -- 172.21.15.100:0/271815029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa350009ae0 con 0x7fa3580b4330 2024-08-17T02:22:14.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa355d9b700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa358159340 0x7fa358169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:14.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa18e489700 1 Processor -- start 2024-08-17T02:22:14.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0b59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c15b570 con 0x7f4e0c15d230 2024-08-17T02:22:14.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c155d10 con 0x7f4e0c0ca0d0 2024-08-17T02:22:14.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c15ac40 con 0x7f4e0c0cb080 2024-08-17T02:22:14.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f487421c700 1 -- 172.21.15.100:0/3175926579 shutdown_connections 2024-08-17T02:22:14.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa18e489700 1 -- start start 2024-08-17T02:22:14.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f55867fc700 1 -- 172.21.15.100:0/2188454451 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590012630 con 0x7f559814fc20 2024-08-17T02:22:14.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 -- 172.21.15.100:0/2188454451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5598097810 con 0x7f559814fc20 2024-08-17T02:22:14.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f559c9d1700 1 -- 172.21.15.100:0/2188454451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5598097dd0 con 0x7f559814fc20 2024-08-17T02:22:14.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f55867fc700 1 -- 172.21.15.100:0/2188454451 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55900127d0 con 0x7f559814fc20 2024-08-17T02:22:14.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 -- 172.21.15.100:0/336342083 shutdown_connections 2024-08-17T02:22:14.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/336342083 >> v2:172.21.15.123:3300/0 conn(0x7fbbec0aa580 0x7fbbec0a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/336342083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155b30 0x7fbbec0a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/336342083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec155760 0x7fbbec0abf80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 -- 172.21.15.100:0/336342083 >> 172.21.15.100:0/336342083 conn(0x7fbbec067810 msgr2=0x7fbbec067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35659c700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3580b4330 0x7fa3580d0810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa358089140 tx=0x7fa350018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 -- 172.21.15.100:0/336342083 shutdown_connections 2024-08-17T02:22:14.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa3467fc700 1 -- 172.21.15.100:0/271815029 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350008b50 con 0x7fa3580b4330 2024-08-17T02:22:14.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 -- 172.21.15.100:0/271815029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3580ac970 con 0x7fa3580b4330 2024-08-17T02:22:14.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa35ccab700 1 -- 172.21.15.100:0/271815029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3580d0d50 con 0x7fa3580b4330 2024-08-17T02:22:14.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f55867fc700 1 -- 172.21.15.100:0/2188454451 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590012b90 con 0x7f559814fc20 2024-08-17T02:22:14.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa3467fc700 1 -- 172.21.15.100:0/271815029 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa350008cb0 con 0x7fa3580b4330 2024-08-17T02:22:14.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa3467fc700 1 -- 172.21.15.100:0/271815029 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3500266b0 con 0x7fa3580b4330 2024-08-17T02:22:14.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fa18e489700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1880d0010 0x7fa188139430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18e489700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa188159040 0x7fa188139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18e489700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa18815cea0 0x7fa1880b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18e489700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880b2830 con 0x7fa188159040 2024-08-17T02:22:14.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18e489700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880b04d0 con 0x7fa1880d0010 2024-08-17T02:22:14.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18e489700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880b07b0 con 0x7fa18815cea0 2024-08-17T02:22:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f15d22ab700 1 -- 172.21.15.100:0/3982185649 shutdown_connections 2024-08-17T02:22:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f15d22ab700 1 --2- 172.21.15.100:0/3982185649 >> v2:172.21.15.100:3300/0 conn(0x7f15cc0afb90 0x7f15cc0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f15d22ab700 1 --2- 172.21.15.100:0/3982185649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc0ad710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f15d22ab700 1 -- 172.21.15.100:0/3982185649 >> 172.21.15.100:0/3982185649 conn(0x7f15cc067890 msgr2=0x7f15cc067c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f559c9d1700 1 -- 172.21.15.100:0/2188454451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557c0052b0 con 0x7f559814fc20 2024-08-17T02:22:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18dc88700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa18815cea0 0x7fa1880b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18dc88700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa18815cea0 0x7fa1880b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:54088/0 (socket says 172.21.15.100:54088) 2024-08-17T02:22:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18dc88700 1 -- 172.21.15.100:0/367827937 learned_addr learned my addr 172.21.15.100:0/367827937 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e09d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0cb080 0x7f4e0c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e09d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0cb080 0x7f4e0c15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:54076/0 (socket says 172.21.15.100:54076) 2024-08-17T02:22:14.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e09d9b700 1 -- 172.21.15.100:0/4184143121 learned_addr learned my addr 172.21.15.100:0/4184143121 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0ad9d700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e0c15d230 0x7f4e0c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0a59c700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e0c0ca0d0 0x7f4e0c162010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0ad9d700 1 -- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0cb080 msgr2=0x7f4e0c15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0ad9d700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0cb080 0x7f4e0c15ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0ad9d700 1 -- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e0c0ca0d0 msgr2=0x7f4e0c162010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0ad9d700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e0c0ca0d0 0x7f4e0c162010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0ad9d700 1 -- 172.21.15.100:0/4184143121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e00009420 con 0x7f4e0c15d230 2024-08-17T02:22:14.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e09d9b700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0cb080 0x7f4e0c15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:14.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7f4e0ad9d700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e0c15d230 0x7f4e0c15bf40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400cb90 tx=0x7f4e040019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18cc86700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa188159040 0x7fa188139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.296+0000 7fbbf2abf700 1 -- 172.21.15.100:0/336342083 wait complete. 2024-08-17T02:22:14.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf2abf700 1 Processor -- start 2024-08-17T02:22:14.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a84e60700 1 -- 172.21.15.100:0/1509785302 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00c8c0 con 0x7f0a8006aff0 2024-08-17T02:22:14.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a84e60700 1 -- 172.21.15.100:0/1509785302 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c00d040 con 0x7f0a8006aff0 2024-08-17T02:22:14.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a84e60700 1 -- 172.21.15.100:0/1509785302 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00cc30 con 0x7f0a8006aff0 2024-08-17T02:22:14.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a86e64700 1 -- 172.21.15.100:0/1509785302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006aff0 msgr2=0x7f0a801556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a86e64700 1 --2- 172.21.15.100:0/1509785302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006aff0 0x7f0a801556f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a800b13e0 tx=0x7f0a7c00a3e0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18cc86700 1 -- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa18815cea0 msgr2=0x7fa1880b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18cc86700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa18815cea0 0x7fa1880b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18cc86700 1 -- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1880d0010 msgr2=0x7fa188139430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-17T02:22:14.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18cc86700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1880d0010 0x7fa188139430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18cc86700 1 -- 172.21.15.100:0/367827937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa178009420 con 0x7fa188159040 2024-08-17T02:22:14.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f55867fc700 1 -- 172.21.15.100:0/2188454451 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f559000a3b0 con 0x7f559814fc20 2024-08-17T02:22:14.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a86e64700 1 -- 172.21.15.100:0/1509785302 shutdown_connections 2024-08-17T02:22:14.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a86e64700 1 --2- 172.21.15.100:0/1509785302 >> v2:172.21.15.123:3300/0 conn(0x7f0a800b1660 0x7f0a80158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a86e64700 1 --2- 172.21.15.100:0/1509785302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006aff0 0x7f0a801556f0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a800b13e0 tx=0x7f0a7c00a3e0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a86e64700 1 --2- 172.21.15.100:0/1509785302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8006e140 0x7f0a8006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f0a86e64700 1 -- 172.21.15.100:0/1509785302 >> 172.21.15.100:0/1509785302 conn(0x7f0a800678d0 msgr2=0x7f0a80067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 -- 172.21.15.100:0/3175926579 wait complete. 2024-08-17T02:22:14.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4dfa7fc700 1 -- 172.21.15.100:0/4184143121 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400a8b0 con 0x7f4e0c15d230 2024-08-17T02:22:14.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4dfa7fc700 1 -- 172.21.15.100:0/4184143121 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0400aa10 con 0x7f4e0c15d230 2024-08-17T02:22:14.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 Processor -- start 2024-08-17T02:22:14.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0c15af20 con 0x7f4e0c15d230 2024-08-17T02:22:14.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa3467fc700 1 -- 172.21.15.100:0/271815029 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa350003040 con 0x7fa3580b4330 2024-08-17T02:22:14.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4dfa7fc700 1 -- 172.21.15.100:0/4184143121 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400add0 con 0x7f4e0c15d230 2024-08-17T02:22:14.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c152410 con 0x7f4e0c15d230 2024-08-17T02:22:14.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f15d22ab700 1 -- 172.21.15.100:0/3982185649 shutdown_connections 2024-08-17T02:22:14.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 -- start start 2024-08-17T02:22:14.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa3467fc700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa3380609b0 0x7fa338062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa3467fc700 1 -- 172.21.15.100:0/271815029 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa35007eb20 con 0x7fa3580b4330 2024-08-17T02:22:14.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf2abf700 1 -- start start 2024-08-17T02:22:14.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf2abf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec0aa580 0x7fbbec0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf2abf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155760 0x7fbbec14cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf2abf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec155b30 0x7fbbec0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf2abf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec14d1a0 con 0x7fbbec0aa580 2024-08-17T02:22:14.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf2abf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec14dc50 con 0x7fbbec155b30 2024-08-17T02:22:14.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf2abf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec14d3b0 con 0x7fbbec155760 2024-08-17T02:22:14.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec0aa580 0x7fbbec0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa18cc86700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa188159040 0x7fa188139970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa178016000 tx=0x7fa178017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec0aa580 0x7fbbec0b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52302/0 (socket says 172.21.15.100:52302) 2024-08-17T02:22:14.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 -- 172.21.15.100:0/3696953965 learned_addr learned my addr 172.21.15.100:0/3696953965 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf105c700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec155b30 0x7fbbec0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbebfff700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155760 0x7fbbec14cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 -- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155760 msgr2=0x7fbbec14cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155760 0x7fbbec14cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 -- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec155b30 msgr2=0x7fbbec0aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec155b30 0x7fbbec0aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 -- 172.21.15.100:0/3696953965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe4009ae0 con 0x7fbbec0aa580 2024-08-17T02:22:14.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa355d9b700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa3380609b0 0x7fa338062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fa35ccab700 1 -- 172.21.15.100:0/271815029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c0052b0 con 0x7fa3580b4330 2024-08-17T02:22:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858003b10 0x7f486c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c152a50 0x7f486c1794a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c1799e0 0x7f486c179e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c17a390 con 0x7f486c152a50 2024-08-17T02:22:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c0af710 con 0x7f4858003b10 2024-08-17T02:22:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f487421c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486c0b19c0 con 0x7f486c1799e0 2024-08-17T02:22:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c15b6e0 con 0x7f4e0c15d230 2024-08-17T02:22:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f48727b9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c1799e0 0x7f486c179e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4871fb8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858003b10 0x7f486c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4871fb8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858003b10 0x7f486c152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:37704/0 (socket says 172.21.15.100:37704) 2024-08-17T02:22:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f48727b9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c1799e0 0x7f486c179e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:54102/0 (socket says 172.21.15.100:54102) 2024-08-17T02:22:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f4871fb8700 1 -- 172.21.15.100:0/1988142053 learned_addr learned my addr 172.21.15.100:0/1988142053 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f48717b7700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c152a50 0x7f486c1794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7fbbf085b700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec0aa580 0x7fbbec0b16c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009fd0 tx=0x7fbbe4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.300+0000 7f15d22ab700 1 -- 172.21.15.100:0/3982185649 wait complete. 2024-08-17T02:22:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f55867fc700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5578060860 0x7f5578062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f48717b7700 1 -- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c1799e0 msgr2=0x7f486c179e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f48717b7700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c1799e0 0x7f486c179e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f48717b7700 1 -- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858003b10 msgr2=0x7f486c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f48717b7700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858003b10 0x7f486c152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f48717b7700 1 -- 172.21.15.100:0/1988142053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c0af960 con 0x7f486c152a50 2024-08-17T02:22:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f55867fc700 1 -- 172.21.15.100:0/2188454451 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f559007c8a0 con 0x7f559814fc20 2024-08-17T02:22:14.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f48717b7700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c152a50 0x7f486c1794a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f485c00d890 tx=0x7f485c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 -- 172.21.15.100:0/1509785302 shutdown_connections 2024-08-17T02:22:14.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f559659c700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5578060860 0x7f5578062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7fa355d9b700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa3380609b0 0x7fa338062e60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa348007fd0 tx=0x7fa348009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 -- 172.21.15.100:0/1509785302 wait complete. 2024-08-17T02:22:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f559659c700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5578060860 0x7f5578062d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f558c0099b0 tx=0x7f558c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d22ab700 1 Processor -- start 2024-08-17T02:22:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d22ab700 1 -- start start 2024-08-17T02:22:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 Processor -- start 2024-08-17T02:22:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 -- start start 2024-08-17T02:22:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006e140 0x7f0a80165190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a800b1660 0x7f0a80154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8013c800 0x7f0a80135c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d22ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15cc0afb90 0x7f15cc0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80138ff0 con 0x7f0a8006e140 2024-08-17T02:22:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d22ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc13e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d22ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15cc13f9b0 0x7f15cc13e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d22ab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0c5f50 con 0x7f15cc0afb90 2024-08-17T02:22:14.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d22ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0c3c20 con 0x7f15cc13f9b0 2024-08-17T02:22:14.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d22ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc16ad70 con 0x7f15cc13f5e0 2024-08-17T02:22:14.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8013d1c0 con 0x7f0a800b1660 2024-08-17T02:22:14.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86e64700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801386c0 con 0x7f0a8013c800 2024-08-17T02:22:14.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15cc0afb90 0x7f15cc0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15cc0afb90 0x7f15cc0c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52318/0 (socket says 172.21.15.100:52318) 2024-08-17T02:22:14.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 -- 172.21.15.100:0/2713125140 learned_addr learned my addr 172.21.15.100:0/2713125140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006e140 0x7f0a80165190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a86663700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8013c800 0x7f0a80135c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006e140 0x7f0a80165190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52316/0 (socket says 172.21.15.100:52316) 2024-08-17T02:22:14.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 -- 172.21.15.100:0/491678651 learned_addr learned my addr 172.21.15.100:0/491678651 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:14.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85661700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a800b1660 0x7f0a80154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15d0848700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15cc13f9b0 0x7f15cc13e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cb7fe700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc13e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 -- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 msgr2=0x7f15cc13e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc13e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 -- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15cc13f9b0 msgr2=0x7f15cc13e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15cc13f9b0 0x7f15cc13e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 -- 172.21.15.100:0/2713125140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc009420 con 0x7f15cc0afb90 2024-08-17T02:22:14.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 -- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8013c800 msgr2=0x7f0a80135c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8013c800 0x7f0a80135c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 -- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a800b1660 msgr2=0x7f0a80154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a800b1660 0x7f0a80154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 -- 172.21.15.100:0/491678651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c00a680 con 0x7f0a8006e140 2024-08-17T02:22:14.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f15cbfff700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15cc0afb90 0x7f15cc0c5a10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0091e0 tx=0x7f15bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f0a85e62700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006e140 0x7f0a80165190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009ef0 tx=0x7f0a7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7f55867fc700 1 -- 172.21.15.100:0/2188454451 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f559004ccf0 con 0x7f559814fc20 2024-08-17T02:22:14.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.304+0000 7fbbe8ff9700 1 -- 172.21.15.100:0/3696953965 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4008910 con 0x7fbbec0aa580 2024-08-17T02:22:14.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbec14d690 con 0x7fbbec0aa580 2024-08-17T02:22:14.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f0a75ffb700 1 -- 172.21.15.100:0/491678651 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7800cc20 con 0x7f0a8006e140 2024-08-17T02:22:14.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec175b40 con 0x7fbbec0aa580 2024-08-17T02:22:14.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7fa17d7fa700 1 -- 172.21.15.100:0/367827937 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800b980 con 0x7fa188159040 2024-08-17T02:22:14.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f4861ffb700 1 -- 172.21.15.100:0/1988142053 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c0125f0 con 0x7f486c152a50 2024-08-17T02:22:14.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486c0afc40 con 0x7f486c152a50 2024-08-17T02:22:14.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486c0b1290 con 0x7f486c152a50 2024-08-17T02:22:14.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1880b0a90 con 0x7fa188159040 2024-08-17T02:22:14.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa188139eb0 con 0x7fa188159040 2024-08-17T02:22:14.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f15b3fff700 1 -- 172.21.15.100:0/2713125140 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00bb20 con 0x7f15cc0afb90 2024-08-17T02:22:14.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc16a9c0 con 0x7f15cc0afb90 2024-08-17T02:22:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc0c35d0 con 0x7f15cc0afb90 2024-08-17T02:22:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a801389a0 con 0x7f0a8006e140 2024-08-17T02:22:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a801656d0 con 0x7f0a8006e140 2024-08-17T02:22:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f4dfa7fc700 1 -- 172.21.15.100:0/4184143121 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4e04007300 con 0x7f4e0c15d230 2024-08-17T02:22:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f4dfa7fc700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4dec060960 0x7f4dec062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f4e0a59c700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4dec060960 0x7f4dec062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486c0afe80 con 0x7f486c152a50 2024-08-17T02:22:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a800643a0 con 0x7f0a8006e140 2024-08-17T02:22:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.308+0000 7f4e0a59c700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4dec060960 0x7f4dec062e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0037c0 tx=0x7f4dfc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec0645c0 con 0x7fbbec0aa580 2024-08-17T02:22:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc0643a0 con 0x7f15cc0afb90 2024-08-17T02:22:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1880b0f90 con 0x7fa188159040 2024-08-17T02:22:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7fa17d7fa700 1 -- 172.21.15.100:0/367827937 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17802e450 con 0x7fa188159040 2024-08-17T02:22:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7fa17d7fa700 1 -- 172.21.15.100:0/367827937 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17802e5e0 con 0x7fa188159040 2024-08-17T02:22:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7f4dfa7fc700 1 -- 172.21.15.100:0/4184143121 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e04052270 con 0x7f4e0c15d230 2024-08-17T02:22:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7f4861ffb700 1 -- 172.21.15.100:0/1988142053 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c012960 con 0x7f486c152a50 2024-08-17T02:22:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7fbbe8ff9700 1 -- 172.21.15.100:0/3696953965 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe4008c80 con 0x7fbbec0aa580 2024-08-17T02:22:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7fbbe8ff9700 1 -- 172.21.15.100:0/3696953965 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4008df0 con 0x7fbbec0aa580 2024-08-17T02:22:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7f0a75ffb700 1 -- 172.21.15.100:0/491678651 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7800f040 con 0x7f0a8006e140 2024-08-17T02:22:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7f4861ffb700 1 -- 172.21.15.100:0/1988142053 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c012ad0 con 0x7f486c152a50 2024-08-17T02:22:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7f0a75ffb700 1 -- 172.21.15.100:0/491678651 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78014480 con 0x7f0a8006e140 2024-08-17T02:22:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7f15b3fff700 1 -- 172.21.15.100:0/2713125140 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc011630 con 0x7f15cc0afb90 2024-08-17T02:22:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7f15b3fff700 1 -- 172.21.15.100:0/2713125140 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0117c0 con 0x7f15cc0afb90 2024-08-17T02:22:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.312+0000 7fa3467fc700 1 -- 172.21.15.100:0/271815029 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa35004ef50 con 0x7fa3580b4330 2024-08-17T02:22:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.316+0000 7f4dfa7fc700 1 -- 172.21.15.100:0/4184143121 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4e0404de70 con 0x7f4e0c15d230 2024-08-17T02:22:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fa17d7fa700 1 -- 172.21.15.100:0/367827937 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa178023070 con 0x7fa188159040 2024-08-17T02:22:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fa17d7fa700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa16c060860 0x7fa16c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fbbe8ff9700 1 -- 172.21.15.100:0/3696953965 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbbe4003040 con 0x7fbbec0aa580 2024-08-17T02:22:14.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f15b3fff700 1 -- 172.21.15.100:0/2713125140 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f15bc011920 con 0x7f15cc0afb90 2024-08-17T02:22:14.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fbbe8ff9700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd8060860 0x7fbbd8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fbbe8ff9700 1 -- 172.21.15.100:0/3696953965 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbbe407e890 con 0x7fbbec0aa580 2024-08-17T02:22:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f15b3fff700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f15b8060860 0x7f15b8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fa18d487700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa16c060860 0x7fa16c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fa18d487700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa16c060860 0x7fa16c062d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa188155c70 tx=0x7fa180001580 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fa17d7fa700 1 -- 172.21.15.100:0/367827937 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa178052740 con 0x7fa188159040 2024-08-17T02:22:14.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f4861ffb700 1 -- 172.21.15.100:0/1988142053 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f485c012c30 con 0x7f486c152a50 2024-08-17T02:22:14.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f4861ffb700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4850060860 0x7f4850062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f4861ffb700 1 -- 172.21.15.100:0/1988142053 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f485c07d6d0 con 0x7f486c152a50 2024-08-17T02:22:14.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f4871fb8700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4850060860 0x7f4850062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f15b3fff700 1 -- 172.21.15.100:0/2713125140 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15bc07e720 con 0x7f15cc0afb90 2024-08-17T02:22:14.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7fbbebfff700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd8060860 0x7fbbd8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f0a75ffb700 1 -- 172.21.15.100:0/491678651 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0a780145e0 con 0x7f0a8006e140 2024-08-17T02:22:14.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f15cb7fe700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f15b8060860 0x7f15b8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f0a75ffb700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a640608e0 0x7f0a64062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f4871fb8700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4850060860 0x7f4850062d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4868007f40 tx=0x7f4868015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f0a75ffb700 1 -- 172.21.15.100:0/491678651 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0a7807db20 con 0x7f0a8006e140 2024-08-17T02:22:14.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f15cb7fe700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f15b8060860 0x7f15b8062d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15cc069a80 tx=0x7f15c4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.320+0000 7f0a85661700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a640608e0 0x7f0a64062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:14.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.324+0000 7fbbebfff700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd8060860 0x7fbbd8062d10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbbec179360 tx=0x7fbbdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.324+0000 7f0a85661700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a640608e0 0x7f0a64062d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a8006efe0 tx=0x7f0a7c018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:14.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.324+0000 7fa17d7fa700 1 -- 172.21.15.100:0/367827937 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa17804e340 con 0x7fa188159040 2024-08-17T02:22:14.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.332+0000 7fbbe8ff9700 1 -- 172.21.15.100:0/3696953965 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbbe404ec40 con 0x7fbbec0aa580 2024-08-17T02:22:14.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.340+0000 7f4861ffb700 1 -- 172.21.15.100:0/1988142053 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f485c04db20 con 0x7f486c152a50 2024-08-17T02:22:14.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.348+0000 7f0a75ffb700 1 -- 172.21.15.100:0/491678651 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0a7804df50 con 0x7f0a8006e140 2024-08-17T02:22:14.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.356+0000 7f15b3fff700 1 -- 172.21.15.100:0/2713125140 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f15bc04ead0 con 0x7f15cc0afb90 2024-08-17T02:22:14.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.588+0000 7f559c9d1700 1 -- 172.21.15.100:0/2188454451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f557c005830 con 0x7f559814fc20 2024-08-17T02:22:14.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.588+0000 7f55867fc700 1 -- 172.21.15.100:0/2188454451 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5590050310 con 0x7f559814fc20 2024-08-17T02:22:14.595 INFO:teuthology.orchestra.run.smithi100.stdout:60129542148 2024-08-17T02:22:14.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 -- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5578060860 msgr2=0x7f5578062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5578060860 0x7f5578062d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f558c0099b0 tx=0x7f558c016000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 -- 172.21.15.100:0/2188454451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559814fc20 msgr2=0x7f55981492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559814fc20 0x7f55981492b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f559000c930 tx=0x7f55900026e0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 -- 172.21.15.100:0/2188454451 shutdown_connections 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5578060860 0x7f5578062d10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559814fc20 0x7f55981492b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f559814f850 0x7f5598171050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 --2- 172.21.15.100:0/2188454451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55980ae5d0 0x7f5598151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 -- 172.21.15.100:0/2188454451 >> 172.21.15.100:0/2188454451 conn(0x7f5598068300 msgr2=0x7f55981544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 -- 172.21.15.100:0/2188454451 shutdown_connections 2024-08-17T02:22:14.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.596+0000 7f5577fff700 1 -- 172.21.15.100:0/2188454451 wait complete. 2024-08-17T02:22:14.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.608+0000 7fa35ccab700 1 -- 172.21.15.100:0/271815029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa33c005560 con 0x7fa3580b4330 2024-08-17T02:22:14.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.608+0000 7fa3467fc700 1 -- 172.21.15.100:0/271815029 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa350021070 con 0x7fa3580b4330 2024-08-17T02:22:14.613 INFO:teuthology.orchestra.run.smithi100.stdout:60129542148 2024-08-17T02:22:14.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.1 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.608+0000 7fa337fff700 1 -- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa3380609b0 msgr2=0x7fa338062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.608+0000 7fa337fff700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa3380609b0 0x7fa338062e60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa348007fd0 tx=0x7fa348009040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.608+0000 7fa337fff700 1 -- 172.21.15.100:0/271815029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3580b4330 msgr2=0x7fa3580d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.608+0000 7fa337fff700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3580b4330 0x7fa3580d0810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa358089140 tx=0x7fa350018000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.612+0000 7fa337fff700 1 -- 172.21.15.100:0/271815029 shutdown_connections 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.612+0000 7fa337fff700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa3380609b0 0x7fa338062e60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.612+0000 7fa337fff700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa358159c50 0x7fa3580c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.612+0000 7fa337fff700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa358159340 0x7fa358169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.612+0000 7fa337fff700 1 --2- 172.21.15.100:0/271815029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3580b4330 0x7fa3580d0810 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.612+0000 7fa337fff700 1 -- 172.21.15.100:0/271815029 >> 172.21.15.100:0/271815029 conn(0x7fa3580994f0 msgr2=0x7fa358068320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.612+0000 7fa337fff700 1 -- 172.21.15.100:0/271815029 shutdown_connections 2024-08-17T02:22:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.612+0000 7fa337fff700 1 -- 172.21.15.100:0/271815029 wait complete. 2024-08-17T02:22:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4e0c0638f0 con 0x7f4e0c15d230 2024-08-17T02:22:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4dfa7fc700 1 -- 172.21.15.100:0/4184143121 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e04051490 con 0x7f4e0c15d230 2024-08-17T02:22:14.618 INFO:teuthology.orchestra.run.smithi100.stdout:60129542148 2024-08-17T02:22:14.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4dec060960 msgr2=0x7f4dec062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4dec060960 0x7f4dec062e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0037c0 tx=0x7f4dfc009040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e0c15d230 msgr2=0x7f4e0c15bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e0c15d230 0x7f4e0c15bf40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400cb90 tx=0x7f4e040019a0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 shutdown_connections 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4dec060960 0x7f4dec062e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e0c15d230 0x7f4e0c15bf40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0c0cb080 0x7f4e0c15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 --2- 172.21.15.100:0/4184143121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e0c0ca0d0 0x7f4e0c162010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.616+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 >> 172.21.15.100:0/4184143121 conn(0x7f4e0c0cd860 msgr2=0x7f4e0c158420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.620+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 shutdown_connections 2024-08-17T02:22:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.620+0000 7f4e0b59e700 1 -- 172.21.15.100:0/4184143121 wait complete. 2024-08-17T02:22:14.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.6 2024-08-17T02:22:14.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.7 2024-08-17T02:22:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.632+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f486c0b17f0 con 0x7f486c152a50 2024-08-17T02:22:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.632+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbbec0ac0b0 con 0x7fbbec0aa580 2024-08-17T02:22:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.632+0000 7f4861ffb700 1 -- 172.21.15.100:0/1988142053 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f485c051140 con 0x7f486c152a50 2024-08-17T02:22:14.636 INFO:teuthology.orchestra.run.smithi100.stdout:60129542148 2024-08-17T02:22:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.632+0000 7fbbe8ff9700 1 -- 172.21.15.100:0/3696953965 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbbe4052260 con 0x7fbbec0aa580 2024-08-17T02:22:14.637 INFO:teuthology.orchestra.run.smithi100.stdout:60129542148 2024-08-17T02:22:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa1880643a0 con 0x7fa188159040 2024-08-17T02:22:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4850060860 msgr2=0x7f4850062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4850060860 0x7f4850062d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4868007f40 tx=0x7f4868015000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c152a50 msgr2=0x7f486c1794a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c152a50 0x7f486c1794a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f485c00d890 tx=0x7f485c008040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 shutdown_connections 2024-08-17T02:22:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4850060860 0x7f4850062d10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c1799e0 0x7f486c179e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c152a50 0x7f486c1794a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 --2- 172.21.15.100:0/1988142053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858003b10 0x7f486c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 >> 172.21.15.100:0/1988142053 conn(0x7f486c067810 msgr2=0x7f486c0aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 shutdown_connections 2024-08-17T02:22:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fa17d7fa700 1 -- 172.21.15.100:0/367827937 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa178051960 con 0x7fa188159040 2024-08-17T02:22:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7f487421c700 1 -- 172.21.15.100:0/1988142053 wait complete. 2024-08-17T02:22:14.641 INFO:teuthology.orchestra.run.smithi100.stdout:60129542148 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd8060860 msgr2=0x7fbbd8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd8060860 0x7fbbd8062d10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbbec179360 tx=0x7fbbdc008040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec0aa580 msgr2=0x7fbbec0b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec0aa580 0x7fbbec0b16c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009fd0 tx=0x7fbbe4018000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 shutdown_connections 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbbd8060860 0x7fbbd8062d10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbec155b30 0x7fbbec0aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbec155760 0x7fbbec14cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 --2- 172.21.15.100:0/3696953965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbec0aa580 0x7fbbec0b16c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 >> 172.21.15.100:0/3696953965 conn(0x7fbbec067810 msgr2=0x7fbbec06d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 shutdown_connections 2024-08-17T02:22:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.636+0000 7fbbf2abf700 1 -- 172.21.15.100:0/3696953965 wait complete. 2024-08-17T02:22:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa16c060860 msgr2=0x7fa16c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa16c060860 0x7fa16c062d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa188155c70 tx=0x7fa180001580 comp rx=0 tx=0).stop 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa188159040 msgr2=0x7fa188139970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa188159040 0x7fa188139970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa178016000 tx=0x7fa178017000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 shutdown_connections 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa16c060860 0x7fa16c062d10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa18815cea0 0x7fa1880b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa188159040 0x7fa188139970 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 --2- 172.21.15.100:0/367827937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1880d0010 0x7fa188139430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 >> 172.21.15.100:0/367827937 conn(0x7fa188068250 msgr2=0x7fa18815b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 shutdown_connections 2024-08-17T02:22:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.640+0000 7fa18e489700 1 -- 172.21.15.100:0/367827937 wait complete. 2024-08-17T02:22:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.644+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0a8006f5f0 con 0x7f0a8006e140 2024-08-17T02:22:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.644+0000 7f0a75ffb700 1 -- 172.21.15.100:0/491678651 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a78051570 con 0x7f0a8006e140 2024-08-17T02:22:14.647 INFO:teuthology.orchestra.run.smithi100.stdout:60129542148 2024-08-17T02:22:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a640608e0 msgr2=0x7f0a64062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a640608e0 0x7f0a64062d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a8006efe0 tx=0x7f0a7c018000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006e140 msgr2=0x7f0a80165190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006e140 0x7f0a80165190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009ef0 tx=0x7f0a7800b040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 shutdown_connections 2024-08-17T02:22:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a640608e0 0x7f0a64062d90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a8013c800 0x7f0a80135c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a800b1660 0x7f0a80154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 --2- 172.21.15.100:0/491678651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8006e140 0x7f0a80165190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 >> 172.21.15.100:0/491678651 conn(0x7f0a800678d0 msgr2=0x7f0a80069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 shutdown_connections 2024-08-17T02:22:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.648+0000 7f0a86e64700 1 -- 172.21.15.100:0/491678651 wait complete. 2024-08-17T02:22:14.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.5 2024-08-17T02:22:14.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.4 2024-08-17T02:22:14.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.2 2024-08-17T02:22:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f15cc06b220 con 0x7f15cc0afb90 2024-08-17T02:22:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15b3fff700 1 -- 172.21.15.100:0/2713125140 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15bc0520f0 con 0x7f15cc0afb90 2024-08-17T02:22:14.658 INFO:teuthology.orchestra.run.smithi100.stdout:60129542148 2024-08-17T02:22:14.659 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.0 2024-08-17T02:22:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f15b8060860 msgr2=0x7f15b8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f15b8060860 0x7f15b8062d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15cc069a80 tx=0x7f15c4009040 comp rx=0 tx=0).stop 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15cc0afb90 msgr2=0x7f15cc0c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15cc0afb90 0x7f15cc0c5a10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0091e0 tx=0x7f15bc017000 comp rx=0 tx=0).stop 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 shutdown_connections 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f15b8060860 0x7f15b8062d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15cc13f9b0 0x7f15cc13e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15cc13f5e0 0x7f15cc13e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 --2- 172.21.15.100:0/2713125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15cc0afb90 0x7f15cc0c5a10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 >> 172.21.15.100:0/2713125140 conn(0x7f15cc067890 msgr2=0x7f15cc0a9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 shutdown_connections 2024-08-17T02:22:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:14.656+0000 7f15d22ab700 1 -- 172.21.15.100:0/2713125140 wait complete. 2024-08-17T02:22:14.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.3 2024-08-17T02:22:15.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-17T02:22:15.628 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-17T02:22:15.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-17T02:22:15.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-17T02:22:15.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-17T02:22:15.657 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-17T02:22:15.660 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-17T02:22:15.671 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-17T02:22:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f75a158b700 1 Processor -- start 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f75a158b700 1 -- start start 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f75a158b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f759c13b630 0x7f759c0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f75a158b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13bf10 0x7f759c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f75a158b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f759c0b1950 0x7f759c06e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f75a158b700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c13cf90 con 0x7f759c0b1950 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f75a158b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c13c380 con 0x7f759c13b630 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f75a158b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c13c660 con 0x7f759c13bf10 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f759c0b1950 0x7f759c06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f759c0b1950 0x7f759c06e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52320/0 (socket says 172.21.15.100:52320) 2024-08-17T02:22:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 -- 172.21.15.100:0/824521547 learned_addr learned my addr 172.21.15.100:0/824521547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 -- 172.21.15.100:0/824521547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13bf10 msgr2=0x7f759c06df10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:22:15.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 --2- 172.21.15.100:0/824521547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13bf10 0x7f759c06df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 -- 172.21.15.100:0/824521547 >> v2:172.21.15.123:3300/0 conn(0x7f759c13b630 msgr2=0x7f759c0b1410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:22:15.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 --2- 172.21.15.100:0/824521547 >> v2:172.21.15.123:3300/0 conn(0x7f759c13b630 0x7f759c0b1410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 -- 172.21.15.100:0/824521547 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c13c940 con 0x7f759c0b1950 2024-08-17T02:22:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f759b7fe700 1 --2- 172.21.15.100:0/824521547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c0b1950 0x7f759c06e450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f759c06c070 tx=0x7f7594007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3beed82d8baa866d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f7599ffb700 1 -- 172.21.15.100:0/824521547 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400de90 con 0x7f759c0b1950 2024-08-17T02:22:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.752+0000 7f7599ffb700 1 -- 172.21.15.100:0/824521547 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7594007b40 con 0x7f759c0b1950 2024-08-17T02:22:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.756+0000 7f75a158b700 1 -- 172.21.15.100:0/824521547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c0b1950 msgr2=0x7f759c06e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.756+0000 7f75a158b700 1 --2- 172.21.15.100:0/824521547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c0b1950 0x7f759c06e450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f759c06c070 tx=0x7f7594007690 comp rx=0 tx=0).stop 2024-08-17T02:22:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 -- 172.21.15.100:0/824521547 shutdown_connections 2024-08-17T02:22:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 --2- 172.21.15.100:0/824521547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c0b1950 0x7f759c06e450 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 --2- 172.21.15.100:0/824521547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13bf10 0x7f759c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 --2- 172.21.15.100:0/824521547 >> v2:172.21.15.123:3300/0 conn(0x7f759c13b630 0x7f759c0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 -- 172.21.15.100:0/824521547 >> 172.21.15.100:0/824521547 conn(0x7f759c0678d0 msgr2=0x7f759c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 -- 172.21.15.100:0/824521547 shutdown_connections 2024-08-17T02:22:15.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 -- 172.21.15.100:0/824521547 wait complete. 2024-08-17T02:22:15.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 Processor -- start 2024-08-17T02:22:15.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 -- start start 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f759c0b1950 0x7f759c16d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13b630 0x7f759c0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c13bf10 0x7f759c0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c12eae0 con 0x7f759c13bf10 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0ca940 con 0x7f759c0b1950 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.760+0000 7f75a158b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0cac20 con 0x7f759c13b630 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c13bf10 0x7f759c0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c13bf10 0x7f759c0ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52334/0 (socket says 172.21.15.100:52334) 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 -- 172.21.15.100:0/1334938149 learned_addr learned my addr 172.21.15.100:0/1334938149 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759affd700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f759c0b1950 0x7f759c16d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759a7fc700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13b630 0x7f759c0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 -- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13b630 msgr2=0x7f759c0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13b630 0x7f759c0c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 -- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f759c0b1950 msgr2=0x7f759c16d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f759c0b1950 0x7f759c16d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 -- 172.21.15.100:0/1334938149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759400be70 con 0x7f759c13bf10 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759affd700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f759c0b1950 0x7f759c16d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f759b7fe700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c13bf10 0x7f759c0ca050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f75940076d0 tx=0x7f759401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f758affd700 1 -- 172.21.15.100:0/1334938149 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594025430 con 0x7f759c13bf10 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f758affd700 1 -- 172.21.15.100:0/1334938149 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7594020070 con 0x7f759c13bf10 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f758affd700 1 -- 172.21.15.100:0/1334938149 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594025800 con 0x7f759c13bf10 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f75a158b700 1 -- 172.21.15.100:0/1334938149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c0caf00 con 0x7f759c13bf10 2024-08-17T02:22:15.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f75a158b700 1 -- 172.21.15.100:0/1334938149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c0ca590 con 0x7f759c13bf10 2024-08-17T02:22:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.764+0000 7f758affd700 1 -- 172.21.15.100:0/1334938149 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7594022030 con 0x7f759c13bf10 2024-08-17T02:22:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.768+0000 7f75a158b700 1 -- 172.21.15.100:0/1334938149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75780052b0 con 0x7f759c13bf10 2024-08-17T02:22:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.768+0000 7f758affd700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7580060860 0x7f7580062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.768+0000 7f758affd700 1 -- 172.21.15.100:0/1334938149 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f759407e2d0 con 0x7f759c13bf10 2024-08-17T02:22:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.768+0000 7f759affd700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7580060860 0x7f7580062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.768+0000 7f759affd700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7580060860 0x7f7580062d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f758c006fd0 tx=0x7f758c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.776+0000 7f758affd700 1 -- 172.21.15.100:0/1334938149 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f759404e720 con 0x7f759c13bf10 2024-08-17T02:22:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b34867700 1 Processor -- start 2024-08-17T02:22:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1b4c2700 1 Processor -- start 2024-08-17T02:22:15.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1b4c2700 1 -- start start 2024-08-17T02:22:15.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b34867700 1 -- start start 2024-08-17T02:22:15.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebf071700 1 Processor -- start 2024-08-17T02:22:15.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a63cc700 1 Processor -- start 2024-08-17T02:22:15.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebf071700 1 -- start start 2024-08-17T02:22:15.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a63cc700 1 -- start start 2024-08-17T02:22:15.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1b4c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1b4c2700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcb140cb390 0x7fcb14160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b34867700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b30158fb0 0x7f7b300aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b34867700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 0x7f7b300ac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b34867700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b300ab2d0 con 0x7f7b3015d910 2024-08-17T02:22:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b34867700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b300ab5b0 con 0x7f7b30158fb0 2024-08-17T02:22:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b34867700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300ab890 con 0x7f7b3006e620 2024-08-17T02:22:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2ed9d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b30158fb0 0x7f7b300aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1b4c2700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb140cbaa0 con 0x7fcb14154900 2024-08-17T02:22:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2ed9d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b30158fb0 0x7f7b300aaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52346/0 (socket says 172.21.15.100:52346) 2024-08-17T02:22:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2ed9d700 1 -- 172.21.15.100:0/1185045311 learned_addr learned my addr 172.21.15.100:0/1185045311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1a4c0700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcb14154900 0x7fcb140b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50892/0 (socket says 172.21.15.100:50892) 2024-08-17T02:22:15.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1a4c0700 1 -- 172.21.15.100:0/2850623105 learned_addr learned my addr 172.21.15.100:0/2850623105 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:15.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebf071700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7eb806e470 0x7f7eb806e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebf071700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb80cad40 0x7f7eb80cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2850623105 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb14068790 con 0x7fcb140cb390 2024-08-17T02:22:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2850623105 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb141583f0 con 0x7fcb140b14a0 2024-08-17T02:22:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb19cbf700 1 --2- 172.21.15.100:0/2850623105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebf071700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb81610c0 con 0x7f7eb80adf10 2024-08-17T02:22:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebf071700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb8158290 con 0x7f7eb806e470 2024-08-17T02:22:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebf071700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb815dff0 con 0x7f7eb80cad40 2024-08-17T02:22:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebce0d700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7eb80adf10 0x7f7eb80ae2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50908/0 (socket says 172.21.15.100:50908) 2024-08-17T02:22:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebce0d700 1 -- 172.21.15.100:0/4255754602 learned_addr learned my addr 172.21.15.100:0/4255754602 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2f59e700 1 --2- 172.21.15.100:0/1185045311 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 0x7f7b300ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a63cc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f50a00b0d20 0x7f50a006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a63cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a006e450 0x7f50a00b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a63cc700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a0141a00 con 0x7f50a00b1630 2024-08-17T02:22:15.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a63cc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a006e8a0 con 0x7f50a00b0d20 2024-08-17T02:22:15.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a63cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0140b10 con 0x7f50a006e450 2024-08-17T02:22:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fb670bc1700 1 Processor -- start 2024-08-17T02:22:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb1acc1700 1 --2- 172.21.15.100:0/2850623105 >> v2:172.21.15.100:3300/0 conn(0x7fcb140cb390 0x7fcb14160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fb670bc1700 1 -- start start 2024-08-17T02:22:15.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2ed9d700 1 -- 172.21.15.100:0/1185045311 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 msgr2=0x7f7b300ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2ed9d700 1 --2- 172.21.15.100:0/1185045311 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 0x7f7b300ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2ed9d700 1 -- 172.21.15.100:0/1185045311 >> v1:172.21.15.123:6789/0 conn(0x7f7b3015d910 legacy=0x7f7b3006df10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:22:15.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2ed9d700 1 -- 172.21.15.100:0/1185045311 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300abb70 con 0x7f7b30158fb0 2024-08-17T02:22:15.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb194be700 1 -- 172.21.15.100:0/2850623105 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910510515 0 0) 0x7fcb140cbaa0 con 0x7fcb14154900 2024-08-17T02:22:15.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb194be700 1 -- 172.21.15.100:0/2850623105 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb00003570 con 0x7fcb14154900 2024-08-17T02:22:15.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499562394 0 0) 0x7f7eb81610c0 con 0x7f7eb80adf10 2024-08-17T02:22:15.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea0003570 con 0x7f7eb80adf10 2024-08-17T02:22:15.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a53ca700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f50a00b0d20 0x7f50a006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a53ca700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f50a00b0d20 0x7f50a006df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:37746/0 (socket says 172.21.15.100:37746) 2024-08-17T02:22:15.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a53ca700 1 -- 172.21.15.100:0/2506188904 learned_addr learned my addr 172.21.15.100:0/2506188904 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb19cbf700 1 -- 172.21.15.100:0/2850623105 >> v2:172.21.15.100:3300/0 conn(0x7fcb140cb390 msgr2=0x7fcb14160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb19cbf700 1 --2- 172.21.15.100:0/2850623105 >> v2:172.21.15.100:3300/0 conn(0x7fcb140cb390 0x7fcb14160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb19cbf700 1 -- 172.21.15.100:0/2850623105 >> v1:172.21.15.100:6789/0 conn(0x7fcb14154900 legacy=0x7fcb140b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb19cbf700 1 -- 172.21.15.100:0/2850623105 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14157630 con 0x7fcb140b14a0 2024-08-17T02:22:15.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb80cad40 0x7f7eb80cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a4bc9700 1 --2- 172.21.15.100:0/2506188904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a006e450 0x7f50a00b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2ed9d700 1 --2- 172.21.15.100:0/1185045311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b30158fb0 0x7f7b300aaa80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7b200099b0 tx=0x7f7b20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ac72105a3b553 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb19cbf700 1 --2- 172.21.15.100:0/2850623105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140cae50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcb040098e0 tx=0x7fcb040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b205b26a60114d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1105239539 0 0) 0x7f7ea0003570 con 0x7f7eb80adf10 2024-08-17T02:22:15.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb80cad40 msgr2=0x7f7eb80cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb80cad40 0x7f7eb80cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 >> v2:172.21.15.100:3300/0 conn(0x7f7eb806e470 msgr2=0x7f7eb806e910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:22:15.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2dd9b700 1 -- 172.21.15.100:0/1185045311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000b820 con 0x7f7b30158fb0 2024-08-17T02:22:15.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2dd9b700 1 -- 172.21.15.100:0/1185045311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2000b980 con 0x7f7b30158fb0 2024-08-17T02:22:15.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7b2dd9b700 1 -- 172.21.15.100:0/1185045311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000bd20 con 0x7f7b30158fb0 2024-08-17T02:22:15.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb7fff700 1 --2- 172.21.15.100:0/4255754602 >> v2:172.21.15.100:3300/0 conn(0x7f7eb806e470 0x7f7eb806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a53ca700 1 -- 172.21.15.100:0/2506188904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a006e450 msgr2=0x7f50a00b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a53ca700 1 --2- 172.21.15.100:0/2506188904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a006e450 0x7f50a00b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a53ca700 1 -- 172.21.15.100:0/2506188904 >> v1:172.21.15.100:6789/0 conn(0x7f50a00b1630 legacy=0x7f50a01610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f50a53ca700 1 -- 172.21.15.100:0/2506188904 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a0140df0 con 0x7f50a00b0d20 2024-08-17T02:22:15.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 --2- 172.21.15.100:0/4255754602 >> v2:172.21.15.100:3300/0 conn(0x7f7eb806e470 0x7f7eb806e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb815e2d0 con 0x7f7eb80adf10 2024-08-17T02:22:15.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7fcb194be700 1 -- 172.21.15.100:0/2850623105 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400b640 con 0x7fcb140b14a0 2024-08-17T02:22:15.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f7eb0001810 con 0x7f7eb80adf10 2024-08-17T02:22:15.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.800+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 >> v1:172.21.15.100:6789/0 conn(0x7f7eb80adf10 legacy=0x7f7eb80ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea0003ae0 0x7f7ea0003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f50a53ca700 1 --2- 172.21.15.100:0/2506188904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a00b0d20 0x7f50a006df10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f509c0099b0 tx=0x7f509c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68014ad7b9ad9bb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb80cad40 0x7f7ea0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb194be700 1 -- 172.21.15.100:0/2850623105 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0400b7a0 con 0x7fcb140b14a0 2024-08-17T02:22:15.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb194be700 1 -- 172.21.15.100:0/2850623105 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400bb20 con 0x7fcb140b14a0 2024-08-17T02:22:15.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0004f00 0x7f7ea000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb815dff0 con 0x7f7ea0004f00 2024-08-17T02:22:15.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea0005640 con 0x7f7ea0003ae0 2024-08-17T02:22:15.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea000bfa0 con 0x7f7eb80cad40 2024-08-17T02:22:15.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb7fff700 1 --2- 172.21.15.100:0/4255754602 >> v2:172.21.15.100:3300/0 conn(0x7f7eb806e470 0x7f7eb806e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:15.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0004f00 0x7f7ea000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebce0d700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea0003ae0 0x7f7ea0003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f5097fff700 1 -- 172.21.15.100:0/2506188904 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c009ea0 con 0x7f50a00b0d20 2024-08-17T02:22:15.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f5097fff700 1 -- 172.21.15.100:0/2506188904 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c00b760 con 0x7f50a00b0d20 2024-08-17T02:22:15.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb670bc1700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb66c155f10 0x7fb66c0aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb670bc1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb66c1562e0 0x7fb66c0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb670bc1700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66c069040 con 0x7fb66c068be0 2024-08-17T02:22:15.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb670bc1700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66c0ac5a0 con 0x7fb66c155f10 2024-08-17T02:22:15.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb670bc1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66c0ab990 con 0x7fb66c1562e0 2024-08-17T02:22:15.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 -- 172.21.15.100:0/1185045311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b30158fb0 msgr2=0x7f7b300aaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 --2- 172.21.15.100:0/1185045311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b30158fb0 0x7f7b300aaa80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7b200099b0 tx=0x7f7b20009250 comp rx=0 tx=0).stop 2024-08-17T02:22:15.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebd60e700 1 -- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb80cad40 msgr2=0x7f7ea0004810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:22:15.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb80cad40 0x7f7ea0004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebd60e700 1 -- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea0003ae0 msgr2=0x7f7ea0003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea0003ae0 0x7f7ea0003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebd60e700 1 -- 172.21.15.100:0/4255754602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0003570 con 0x7f7ea0004f00 2024-08-17T02:22:15.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f5097fff700 1 -- 172.21.15.100:0/2506188904 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c00bb00 con 0x7f50a00b0d20 2024-08-17T02:22:15.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2850623105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 msgr2=0x7fcb140cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2850623105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140cae50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcb040098e0 tx=0x7fcb040091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66a59c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb66c155f10 0x7fb66c0aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66a59c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb66c155f10 0x7fb66c0aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52388/0 (socket says 172.21.15.100:52388) 2024-08-17T02:22:15.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66a59c700 1 -- 172.21.15.100:0/3325062063 learned_addr learned my addr 172.21.15.100:0/3325062063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2850623105 shutdown_connections 2024-08-17T02:22:15.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2850623105 >> v2:172.21.15.100:3300/0 conn(0x7fcb140cb390 0x7fcb14160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2850623105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140cae50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 -- 172.21.15.100:0/1185045311 shutdown_connections 2024-08-17T02:22:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 --2- 172.21.15.100:0/1185045311 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 0x7f7b300ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 --2- 172.21.15.100:0/1185045311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b30158fb0 0x7f7b300aaa80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 -- 172.21.15.100:0/1185045311 >> 172.21.15.100:0/1185045311 conn(0x7f7b300678d0 msgr2=0x7f7b30067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2850623105 >> 172.21.15.100:0/2850623105 conn(0x7fcb140678d0 msgr2=0x7fcb14067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604611886 0 0) 0x7fb660001110 con 0x7fb66c068be0 2024-08-17T02:22:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb654003570 con 0x7fb66c068be0 2024-08-17T02:22:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb669d9b700 1 --2- 172.21.15.100:0/3325062063 >> v2:172.21.15.123:3300/0 conn(0x7fb66c1562e0 0x7fb66c0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 -- 172.21.15.100:0/1185045311 shutdown_connections 2024-08-17T02:22:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 -- 172.21.15.100:0/1185045311 wait complete. 2024-08-17T02:22:15.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 508045984 0 0) 0x7fb654003570 con 0x7fb66c068be0 2024-08-17T02:22:15.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 >> v2:172.21.15.123:3300/0 conn(0x7fb66c1562e0 msgr2=0x7fb66c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 --2- 172.21.15.100:0/3325062063 >> v2:172.21.15.123:3300/0 conn(0x7fb66c1562e0 0x7fb66c0686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 >> v2:172.21.15.100:3300/0 conn(0x7fb66c155f10 msgr2=0x7fb66c0aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 --2- 172.21.15.100:0/3325062063 >> v2:172.21.15.100:3300/0 conn(0x7fb66c155f10 0x7fb66c0aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c0abc70 con 0x7fb66c068be0 2024-08-17T02:22:15.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f50a63cc700 1 -- 172.21.15.100:0/2506188904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a00b0d20 msgr2=0x7f50a006df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f50a63cc700 1 --2- 172.21.15.100:0/2506188904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a00b0d20 0x7f50a006df10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f509c0099b0 tx=0x7f509c009250 comp rx=0 tx=0).stop 2024-08-17T02:22:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7fb660001600 con 0x7fb66c068be0 2024-08-17T02:22:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 >> v1:172.21.15.100:6789/0 conn(0x7fb66c068be0 legacy=0x7fb66c0ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb654003a60 0x7fb654003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654004810 0x7fb654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6540051c0 0x7fb65400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0ac5a0 con 0x7fb6540051c0 2024-08-17T02:22:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0ab990 con 0x7fb654004810 2024-08-17T02:22:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65400c080 con 0x7fb654003a60 2024-08-17T02:22:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2850623105 shutdown_connections 2024-08-17T02:22:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66ad9d700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6540051c0 0x7fb65400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb669d9b700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654004810 0x7fb654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66a59c700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb654003a60 0x7fb654003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0004f00 0x7f7ea000ba60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800aae0 tx=0x7f7ea8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268f001e52660d85 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66ad9d700 1 -- 172.21.15.100:0/3325062063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb654003a60 msgr2=0x7fb654003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66ad9d700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb654003a60 0x7fb654003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66ad9d700 1 -- 172.21.15.100:0/3325062063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654004810 msgr2=0x7fb654004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66ad9d700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654004810 0x7fb654004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66ad9d700 1 -- 172.21.15.100:0/3325062063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb654003570 con 0x7fb6540051c0 2024-08-17T02:22:15.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66ad9d700 1 -- 172.21.15.100:0/3325062063 reap_dead start 2024-08-17T02:22:15.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800c600 con 0x7f7ea0004f00 2024-08-17T02:22:15.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea800c760 con 0x7f7ea0004f00 2024-08-17T02:22:15.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7eb77fe700 1 -- 172.21.15.100:0/4255754602 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800cb20 con 0x7f7ea0004f00 2024-08-17T02:22:15.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 Processor -- start 2024-08-17T02:22:15.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66ad9d700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6540051c0 0x7fb65400ba60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c410 tx=0x7fb66c0ac6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bec4e195e6ffc2 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 -- start start 2024-08-17T02:22:15.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660007ca0 con 0x7fb6540051c0 2024-08-17T02:22:15.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fb66959a700 1 -- 172.21.15.100:0/3325062063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66000cd40 con 0x7fb6540051c0 2024-08-17T02:22:15.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2850623105 wait complete. 2024-08-17T02:22:15.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebf071700 1 -- 172.21.15.100:0/4255754602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0004f00 msgr2=0x7f7ea000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebf071700 1 --2- 172.21.15.100:0/4255754602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0004f00 0x7f7ea000ba60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800aae0 tx=0x7f7ea8008040 comp rx=0 tx=0).stop 2024-08-17T02:22:15.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7ebd60e700 1 -- 172.21.15.100:0/4255754602 reap_dead start 2024-08-17T02:22:15.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 0x7f7b300c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b30158fb0 0x7f7b301697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.804+0000 7f7b34867700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3015d910 0x7f7b30169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b34867700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300aec80 con 0x7f7b3015d910 2024-08-17T02:22:15.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b34867700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30152430 con 0x7f7b30158fb0 2024-08-17T02:22:15.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b34867700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300c8f80 con 0x7f7b3006e620 2024-08-17T02:22:15.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 -- 172.21.15.100:0/4255754602 shutdown_connections 2024-08-17T02:22:15.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 -- 172.21.15.100:0/4255754602 >> 172.21.15.100:0/4255754602 conn(0x7f7eb8067810 msgr2=0x7f7eb8067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 -- 172.21.15.100:0/2506188904 shutdown_connections 2024-08-17T02:22:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 --2- 172.21.15.100:0/2506188904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a006e450 0x7f50a00b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 --2- 172.21.15.100:0/2506188904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a00b0d20 0x7f50a006df10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 -- 172.21.15.100:0/2506188904 >> 172.21.15.100:0/2506188904 conn(0x7f50a00678d0 msgr2=0x7f50a0067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fb670bc1700 1 -- 172.21.15.100:0/3325062063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6540051c0 msgr2=0x7fb65400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fb670bc1700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6540051c0 0x7fb65400ba60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c410 tx=0x7fb66c0ac6e0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3015d910 0x7f7b30169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3015d910 0x7f7b30169d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52404/0 (socket says 172.21.15.100:52404) 2024-08-17T02:22:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 -- 172.21.15.100:0/3125434479 learned_addr learned my addr 172.21.15.100:0/3125434479 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2e59c700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b30158fb0 0x7f7b301697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2ed9d700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 0x7f7b300c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 -- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 msgr2=0x7f7b300c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 0x7f7b300c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 -- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b30158fb0 msgr2=0x7f7b301697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b30158fb0 0x7f7b301697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 -- 172.21.15.100:0/3125434479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200094f0 con 0x7f7b3015d910 2024-08-17T02:22:15.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b2f59e700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3015d910 0x7f7b30169d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400d8c0 tx=0x7f7b24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1b4c2700 1 Processor -- start 2024-08-17T02:22:15.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 -- 172.21.15.100:0/2506188904 shutdown_connections 2024-08-17T02:22:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1b4c2700 1 -- start start 2024-08-17T02:22:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 -- 172.21.15.100:0/2506188904 wait complete. 2024-08-17T02:22:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 -- 172.21.15.100:0/4255754602 shutdown_connections 2024-08-17T02:22:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 -- 172.21.15.100:0/4255754602 wait complete. 2024-08-17T02:22:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2efb110700 1 Processor -- start 2024-08-17T02:22:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b1effd700 1 -- 172.21.15.100:0/3125434479 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b240125f0 con 0x7f7b3015d910 2024-08-17T02:22:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b1effd700 1 -- 172.21.15.100:0/3125434479 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24012750 con 0x7f7b3015d910 2024-08-17T02:22:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b1effd700 1 -- 172.21.15.100:0/3125434479 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24012b10 con 0x7f7b3015d910 2024-08-17T02:22:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b300c9260 con 0x7f7b3015d910 2024-08-17T02:22:15.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2efb110700 1 -- start start 2024-08-17T02:22:15.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1b4c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1b4c2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb140cb390 0x7fcb140b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1b4c2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb14154900 0x7fcb140a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1b4c2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140b12a0 con 0x7fcb140cb390 2024-08-17T02:22:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1b4c2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140a7780 con 0x7fcb14154900 2024-08-17T02:22:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1b4c2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140a6890 con 0x7fcb140b14a0 2024-08-17T02:22:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b300c9820 con 0x7f7b3015d910 2024-08-17T02:22:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fb670bc1700 1 -- 172.21.15.100:0/3325062063 shutdown_connections 2024-08-17T02:22:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fb670bc1700 1 --2- 172.21.15.100:0/3325062063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6540051c0 0x7fb65400ba60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fb670bc1700 1 -- 172.21.15.100:0/3325062063 >> 172.21.15.100:0/3325062063 conn(0x7fb66c06db30 msgr2=0x7fb66c06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 Processor -- start 2024-08-17T02:22:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 -- start start 2024-08-17T02:22:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140a7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:54184/0 (socket says 172.21.15.100:54184) 2024-08-17T02:22:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 -- 172.21.15.100:0/2430213887 learned_addr learned my addr 172.21.15.100:0/2430213887 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2efb110700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2ef414f850 0x7f2ef40691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2efb110700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ef414fc20 0x7f2ef4156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2efb110700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef4069a40 con 0x7f2ef406f2b0 2024-08-17T02:22:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2efb110700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef4069d20 con 0x7f2ef414fc20 2024-08-17T02:22:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2efb110700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef406a000 con 0x7f2ef414f850 2024-08-17T02:22:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b300aee30 con 0x7f7b3015d910 2024-08-17T02:22:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fb670bc1700 1 -- 172.21.15.100:0/3325062063 shutdown_connections 2024-08-17T02:22:15.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 -- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb14154900 msgr2=0x7fcb140a6350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-17T02:22:15.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb14154900 0x7fcb140a6350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 -- 172.21.15.100:0/2430213887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb140cb390 msgr2=0x7fcb140b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb140cb390 0x7fcb140b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 -- 172.21.15.100:0/2430213887 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb04009420 con 0x7fcb140b14a0 2024-08-17T02:22:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2ef3fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ef414fc20 0x7f2ef4156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2ef3fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ef414fc20 0x7f2ef4156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52432/0 (socket says 172.21.15.100:52432) 2024-08-17T02:22:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2ef3fff700 1 -- 172.21.15.100:0/3755338792 learned_addr learned my addr 172.21.15.100:0/3755338792 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f2ef8eac700 1 --2- 172.21.15.100:0/3755338792 >> v2:172.21.15.123:3300/0 conn(0x7f2ef414f850 0x7f2ef40691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea0004f00 0x7f7eb8157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7eb816b8e0 0x7f7eb816ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb816b050 0x7f7eb80a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80a97d0 con 0x7f7eb816b8e0 2024-08-17T02:22:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8157650 con 0x7f7ea0004f00 2024-08-17T02:22:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7ebf071700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8098f40 con 0x7f7eb816b050 2024-08-17T02:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7fcb1a4c0700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140a7c10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c790 tx=0x7fcb10007df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f2ef3fff700 1 -- 172.21.15.100:0/3755338792 >> v2:172.21.15.123:3300/0 conn(0x7f2ef414f850 msgr2=0x7f2ef40691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f2ef3fff700 1 --2- 172.21.15.100:0/3755338792 >> v2:172.21.15.123:3300/0 conn(0x7f2ef414f850 0x7f2ef40691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f2ef3fff700 1 -- 172.21.15.100:0/3755338792 >> v1:172.21.15.123:6789/0 conn(0x7f2ef406f2b0 legacy=0x7f2ef406a7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f2ef3fff700 1 -- 172.21.15.100:0/3755338792 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef4155ce0 con 0x7f2ef414fc20 2024-08-17T02:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f2ef3fff700 1 --2- 172.21.15.100:0/3755338792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef414fc20 0x7f2ef4156200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40705d0 tx=0x7f2ee800a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be0d62e1c981331 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7eb7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7eb816b8e0 0x7f7eb816ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f7eb7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7eb816b8e0 0x7f7eb816ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52448/0 (socket says 172.21.15.100:52448) 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7eb7fff700 1 -- 172.21.15.100:0/295322474 learned_addr learned my addr 172.21.15.100:0/295322474 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb816b050 0x7f7eb80a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebd60e700 1 -- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea0004f00 msgr2=0x7f7eb8157110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea0004f00 0x7f7eb8157110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebd60e700 1 -- 172.21.15.100:0/295322474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7eb816b8e0 msgr2=0x7f7eb816ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7eb816b8e0 0x7f7eb816ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebd60e700 1 -- 172.21.15.100:0/295322474 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb8099220 con 0x7f7eb816b050 2024-08-17T02:22:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebd60e700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb816b050 0x7f7eb80a9c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8016000 tx=0x7f7ea8019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7fcb0a7fc700 1 -- 172.21.15.100:0/2430213887 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10005b30 con 0x7fcb140b14a0 2024-08-17T02:22:15.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7fcb0a7fc700 1 -- 172.21.15.100:0/2430213887 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10005c90 con 0x7fcb140b14a0 2024-08-17T02:22:15.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7fcb0a7fc700 1 -- 172.21.15.100:0/2430213887 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10007300 con 0x7fcb140b14a0 2024-08-17T02:22:15.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb140a6e50 con 0x7fcb140b14a0 2024-08-17T02:22:15.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb140a7410 con 0x7fcb140b14a0 2024-08-17T02:22:15.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7eb4ff9700 1 -- 172.21.15.100:0/295322474 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea801f070 con 0x7f7eb816b050 2024-08-17T02:22:15.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7eb4ff9700 1 -- 172.21.15.100:0/295322474 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea8011600 con 0x7f7eb816b050 2024-08-17T02:22:15.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7eb4ff9700 1 -- 172.21.15.100:0/295322474 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800c600 con 0x7f7eb816b050 2024-08-17T02:22:15.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb80a9180 con 0x7f7eb816b050 2024-08-17T02:22:15.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7b1effd700 1 -- 172.21.15.100:0/3125434479 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7b24013480 con 0x7f7b3015d910 2024-08-17T02:22:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb816b4c0 con 0x7f7eb816b050 2024-08-17T02:22:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7b1effd700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7b140609b0 0x7f7b14062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7b1effd700 1 -- 172.21.15.100:0/3125434479 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7b2407d730 con 0x7f7b3015d910 2024-08-17T02:22:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7b2ed9d700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7b140609b0 0x7f7b14062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7fcb0a7fc700 1 -- 172.21.15.100:0/2430213887 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcb10007670 con 0x7fcb140b14a0 2024-08-17T02:22:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7b2ed9d700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7b140609b0 0x7f7b14062e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b200099b0 tx=0x7f7b20016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7fcb0a7fc700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcaf8060930 0x7fcaf8062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7fcb0a7fc700 1 -- 172.21.15.100:0/2430213887 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb1007c0e0 con 0x7fcb140b14a0 2024-08-17T02:22:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f7eb4ff9700 1 -- 172.21.15.100:0/295322474 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7ea8011c70 con 0x7f7eb816b050 2024-08-17T02:22:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f7eb4ff9700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7ea4060890 0x7f7ea4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 Processor -- start 2024-08-17T02:22:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 -- start start 2024-08-17T02:22:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.808+0000 7f50a63cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a006e450 0x7f50a016e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a63cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a00b0d20 0x7f50a015fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a63cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a00b1630 0x7f50a0160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a63cc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a016fce0 con 0x7f50a00b1630 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a63cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a016f0d0 con 0x7f50a006e450 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a63cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a016f3b0 con 0x7f50a00b0d20 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a00b1630 0x7f50a0160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a00b1630 0x7f50a0160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52460/0 (socket says 172.21.15.100:52460) 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 -- 172.21.15.100:0/46645999 learned_addr learned my addr 172.21.15.100:0/46645999 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a53ca700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a006e450 0x7f50a016e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a4bc9700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a00b0d20 0x7f50a015fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 -- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a00b0d20 msgr2=0x7f50a015fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a00b0d20 0x7f50a015fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 -- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a006e450 msgr2=0x7f50a016e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a006e450 0x7f50a016e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 -- 172.21.15.100:0/46645999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0094f0 con 0x7f50a00b1630 2024-08-17T02:22:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.812+0000 7f50a5bcb700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a00b1630 0x7f50a0160270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f509800cb90 tx=0x7f5098001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 -- 172.21.15.100:0/3325062063 wait complete. 2024-08-17T02:22:15.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb140643a0 con 0x7fcb140b14a0 2024-08-17T02:22:15.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f7eb4ff9700 1 -- 172.21.15.100:0/295322474 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ea807dd70 con 0x7f7eb816b050 2024-08-17T02:22:15.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fcb19cbf700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcaf8060930 0x7fcaf8062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f7ebce0d700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7ea4060890 0x7f7ea4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f7ebce0d700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7ea4060890 0x7f7ea4062d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000a0e0 tx=0x7f7eb0002180 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fcb19cbf700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcaf8060930 0x7fcaf8062de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb040098e0 tx=0x7fcb04017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 Processor -- start 2024-08-17T02:22:15.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 -- start start 2024-08-17T02:22:15.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f50957fa700 1 -- 172.21.15.100:0/46645999 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800a7a0 con 0x7f50a00b1630 2024-08-17T02:22:15.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f7b1effd700 1 -- 172.21.15.100:0/3125434479 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7b2404dbe0 con 0x7f7b3015d910 2024-08-17T02:22:15.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f50957fa700 1 -- 172.21.15.100:0/46645999 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509800a900 con 0x7f50a00b1630 2024-08-17T02:22:15.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a016f690 con 0x7f50a00b1630 2024-08-17T02:22:15.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a016baa0 con 0x7f50a00b1630 2024-08-17T02:22:15.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f50957fa700 1 -- 172.21.15.100:0/46645999 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800aca0 con 0x7f50a00b1630 2024-08-17T02:22:15.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6540051c0 0x7fb66c168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb66c0a9d00 0x7fb66c1418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb66c0aa170 0x7fb66c0af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0ab260 con 0x7fb66c0aa170 2024-08-17T02:22:15.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2ef37fe700 1 -- 172.21.15.100:0/3755338792 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800ca00 con 0x7f2ef414fc20 2024-08-17T02:22:15.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2ef37fe700 1 -- 172.21.15.100:0/3755338792 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee800cb60 con 0x7f2ef414fc20 2024-08-17T02:22:15.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2ef37fe700 1 -- 172.21.15.100:0/3755338792 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80183f0 con 0x7f2ef414fc20 2024-08-17T02:22:15.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0aa650 con 0x7fb6540051c0 2024-08-17T02:22:15.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb670bc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0aa930 con 0x7fb66c0a9d00 2024-08-17T02:22:15.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb66a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6540051c0 0x7fb66c168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2efb110700 1 -- 172.21.15.100:0/3755338792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef414fc20 msgr2=0x7f2ef4156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2efb110700 1 --2- 172.21.15.100:0/3755338792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef414fc20 0x7f2ef4156200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40705d0 tx=0x7f2ee800a3f0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb669d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb66c0a9d00 0x7fb66c1418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb66a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6540051c0 0x7fb66c168ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:37830/0 (socket says 172.21.15.100:37830) 2024-08-17T02:22:15.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb66a59c700 1 -- 172.21.15.100:0/853515707 learned_addr learned my addr 172.21.15.100:0/853515707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb66ad9d700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb66c0aa170 0x7fb66c0af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50800052b0 con 0x7f50a00b1630 2024-08-17T02:22:15.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2efb110700 1 -- 172.21.15.100:0/3755338792 shutdown_connections 2024-08-17T02:22:15.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2efb110700 1 --2- 172.21.15.100:0/3755338792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef414fc20 0x7f2ef4156200 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2efb110700 1 --2- 172.21.15.100:0/3755338792 >> v2:172.21.15.123:3300/0 conn(0x7f2ef414f850 0x7f2ef40691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7f2efb110700 1 -- 172.21.15.100:0/3755338792 >> 172.21.15.100:0/3755338792 conn(0x7f2ef4068300 msgr2=0x7f2ef40a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:15.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb669d9b700 1 -- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6540051c0 msgr2=0x7fb66c168ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb669d9b700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6540051c0 0x7fb66c168ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb669d9b700 1 -- 172.21.15.100:0/853515707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb66c0aa170 msgr2=0x7fb66c0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb669d9b700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb66c0aa170 0x7fb66c0af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.816+0000 7fb669d9b700 1 -- 172.21.15.100:0/853515707 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c0aac10 con 0x7fb66c0a9d00 2024-08-17T02:22:15.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.820+0000 7fb669d9b700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb66c0a9d00 0x7fb66c1418c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb664009e00 tx=0x7fb66400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.820+0000 7f50957fa700 1 -- 172.21.15.100:0/46645999 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5098003030 con 0x7f50a00b1630 2024-08-17T02:22:15.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.820+0000 7f50957fa700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5084060860 0x7f5084062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.820+0000 7f50957fa700 1 -- 172.21.15.100:0/46645999 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f509807d3d0 con 0x7f50a00b1630 2024-08-17T02:22:15.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.820+0000 7f50a53ca700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5084060860 0x7f5084062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.820+0000 7f50a53ca700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5084060860 0x7f5084062d10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f509c0099b0 tx=0x7f509c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 -- 172.21.15.100:0/3755338792 shutdown_connections 2024-08-17T02:22:15.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 -- 172.21.15.100:0/3755338792 wait complete. 2024-08-17T02:22:15.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6248f0b700 1 Processor -- start 2024-08-17T02:22:15.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb8098f40 con 0x7f7eb816b050 2024-08-17T02:22:15.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6248f0b700 1 -- start start 2024-08-17T02:22:15.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f50957fa700 1 -- 172.21.15.100:0/46645999 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5098051330 con 0x7f50a00b1630 2024-08-17T02:22:15.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 Processor -- start 2024-08-17T02:22:15.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 -- start start 2024-08-17T02:22:15.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6248f0b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6244156d80 0x7f6244157150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6248f0b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f62440b0d90 0x7f62440b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6248f0b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62440ad750 0x7f62440adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6248f0b700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62441580e0 con 0x7f62440b0d90 2024-08-17T02:22:15.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6248f0b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62440b1720 con 0x7f6244156d80 2024-08-17T02:22:15.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6248f0b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62441577b0 con 0x7f62440ad750 2024-08-17T02:22:15.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f62440b0d90 0x7f62440b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f62440b0d90 0x7f62440b11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52480/0 (socket says 172.21.15.100:52480) 2024-08-17T02:22:15.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 -- 172.21.15.100:0/2891044286 learned_addr learned my addr 172.21.15.100:0/2891044286 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f62437fe700 1 --2- 172.21.15.100:0/2891044286 >> v2:172.21.15.123:3300/0 conn(0x7f6244156d80 0x7f6244157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6243fff700 1 --2- 172.21.15.100:0/2891044286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62440ad750 0x7f62440adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 -- 172.21.15.100:0/2891044286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62440ad750 msgr2=0x7f62440adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 --2- 172.21.15.100:0/2891044286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62440ad750 0x7f62440adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 -- 172.21.15.100:0/2891044286 >> v2:172.21.15.123:3300/0 conn(0x7f6244156d80 msgr2=0x7f6244157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:15.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 --2- 172.21.15.100:0/2891044286 >> v2:172.21.15.123:3300/0 conn(0x7f6244156d80 0x7f6244157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 -- 172.21.15.100:0/2891044286 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244157a90 con 0x7f62440b0d90 2024-08-17T02:22:15.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef406f2b0 0x7f2ef41484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef414f850 0x7f2ef40afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ef414fc20 0x7f2ef416ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef41697e0 con 0x7f2ef406f2b0 2024-08-17T02:22:15.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef4148a10 con 0x7f2ef414f850 2024-08-17T02:22:15.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2efb110700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef4159ad0 con 0x7f2ef414fc20 2024-08-17T02:22:15.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f6242ffd700 1 --2- 172.21.15.100:0/2891044286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f62440b11e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62440ae7c0 tx=0x7f6234009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607c32b071d86216 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef406f2b0 0x7f2ef41484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef406f2b0 0x7f2ef41484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52488/0 (socket says 172.21.15.100:52488) 2024-08-17T02:22:15.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 -- 172.21.15.100:0/3369419719 learned_addr learned my addr 172.21.15.100:0/3369419719 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7fb65a7fc700 1 -- 172.21.15.100:0/853515707 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66400e040 con 0x7fb66c0a9d00 2024-08-17T02:22:15.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66c0aaef0 con 0x7fb66c0a9d00 2024-08-17T02:22:15.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66c142020 con 0x7fb66c0a9d00 2024-08-17T02:22:15.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 -- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ef414fc20 msgr2=0x7f2ef416ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef96ad700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ef414fc20 0x7f2ef416ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ef414fc20 0x7f2ef416ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 -- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef414f850 msgr2=0x7f2ef40afb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-17T02:22:15.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef414f850 0x7f2ef40afb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7fcb0a7fc700 1 -- 172.21.15.100:0/2430213887 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcb1004c5b0 con 0x7fcb140b14a0 2024-08-17T02:22:15.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 -- 172.21.15.100:0/3369419719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee800a690 con 0x7f2ef406f2b0 2024-08-17T02:22:15.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef96ad700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ef414fc20 0x7f2ef416ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:15.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.824+0000 7f2ef8eac700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef406f2b0 0x7f2ef41484d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00e190 tx=0x7f2eec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2ef0ff9700 1 -- 172.21.15.100:0/3369419719 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00ee30 con 0x7f2ef406f2b0 2024-08-17T02:22:15.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2ef0ff9700 1 -- 172.21.15.100:0/3369419719 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec014420 con 0x7f2ef406f2b0 2024-08-17T02:22:15.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2ef0ff9700 1 -- 172.21.15.100:0/3369419719 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec0147c0 con 0x7f2ef406f2b0 2024-08-17T02:22:15.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef4159db0 con 0x7f2ef406f2b0 2024-08-17T02:22:15.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef40a94c0 con 0x7f2ef406f2b0 2024-08-17T02:22:15.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f62427fc700 1 -- 172.21.15.100:0/2891044286 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234009e70 con 0x7f62440b0d90 2024-08-17T02:22:15.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f62427fc700 1 -- 172.21.15.100:0/2891044286 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623400b860 con 0x7f62440b0d90 2024-08-17T02:22:15.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f62427fc700 1 -- 172.21.15.100:0/2891044286 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623400bc00 con 0x7f62440b0d90 2024-08-17T02:22:15.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f6248f0b700 1 -- 172.21.15.100:0/2891044286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 msgr2=0x7f62440b11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f6248f0b700 1 --2- 172.21.15.100:0/2891044286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f62440b11e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62440ae7c0 tx=0x7f6234009250 comp rx=0 tx=0).stop 2024-08-17T02:22:15.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66c0ab3d0 con 0x7fb66c0a9d00 2024-08-17T02:22:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f6248f0b700 1 -- 172.21.15.100:0/2891044286 shutdown_connections 2024-08-17T02:22:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f6248f0b700 1 --2- 172.21.15.100:0/2891044286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62440ad750 0x7f62440adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f6248f0b700 1 --2- 172.21.15.100:0/2891044286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f62440b11e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f6248f0b700 1 --2- 172.21.15.100:0/2891044286 >> v2:172.21.15.123:3300/0 conn(0x7f6244156d80 0x7f6244157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f6248f0b700 1 -- 172.21.15.100:0/2891044286 >> 172.21.15.100:0/2891044286 conn(0x7f62440678d0 msgr2=0x7f6244067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2ef0ff9700 1 -- 172.21.15.100:0/3369419719 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2eec014b70 con 0x7f2ef406f2b0 2024-08-17T02:22:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2ef0ff9700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2edc060a10 0x7f2edc062ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2ef0ff9700 1 -- 172.21.15.100:0/3369419719 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2eec07d390 con 0x7f2ef406f2b0 2024-08-17T02:22:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.828+0000 7f2ef3fff700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2edc060a10 0x7f2edc062ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.832+0000 7f2ef3fff700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2edc060a10 0x7f2edc062ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40705d0 tx=0x7f2ee8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.832+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef40a9320 con 0x7f2ef406f2b0 2024-08-17T02:22:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.832+0000 7fb65a7fc700 1 -- 172.21.15.100:0/853515707 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66400f040 con 0x7fb66c0a9d00 2024-08-17T02:22:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.832+0000 7fb65a7fc700 1 -- 172.21.15.100:0/853515707 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66400fe70 con 0x7fb66c0a9d00 2024-08-17T02:22:15.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.832+0000 7f7eb4ff9700 1 -- 172.21.15.100:0/295322474 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7ea804e1c0 con 0x7f7eb816b050 2024-08-17T02:22:15.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.832+0000 7fb65a7fc700 1 -- 172.21.15.100:0/853515707 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb664028480 con 0x7fb66c0a9d00 2024-08-17T02:22:15.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.836+0000 7fb65a7fc700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb654066420 0x7fb6540688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.836+0000 7fb65a7fc700 1 -- 172.21.15.100:0/853515707 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb66407dab0 con 0x7fb66c0a9d00 2024-08-17T02:22:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.836+0000 7fb66a59c700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb654066420 0x7fb6540688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.836+0000 7fb66a59c700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb654066420 0x7fb6540688d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009cd0 tx=0x7fb65c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.836+0000 7fb65a7fc700 1 -- 172.21.15.100:0/853515707 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb66404de60 con 0x7fb66c0a9d00 2024-08-17T02:22:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 -- 172.21.15.100:0/2891044286 shutdown_connections 2024-08-17T02:22:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 -- 172.21.15.100:0/2891044286 wait complete. 2024-08-17T02:22:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f2ef0ff9700 1 -- 172.21.15.100:0/3369419719 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2ef40a9320 con 0x7f2ef406f2b0 2024-08-17T02:22:15.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 Processor -- start 2024-08-17T02:22:15.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 -- start start 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62440ad750 0x7f624414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f624414a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6242ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f624414a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6242ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f624414a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52492/0 (socket says 172.21.15.100:52492) 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6244156d80 0x7f624413fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440ca4d0 con 0x7f62440b0d90 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440ca7b0 con 0x7f62440ad750 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6248f0b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62440caa90 con 0x7f6244156d80 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.840+0000 7f6242ffd700 1 -- 172.21.15.100:0/4110074397 learned_addr learned my addr 172.21.15.100:0/4110074397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.844+0000 7f6243fff700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6244156d80 0x7f624413fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.844+0000 7f6242ffd700 1 -- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6244156d80 msgr2=0x7f624413fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.844+0000 7f6242ffd700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6244156d80 0x7f624413fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.844+0000 7f6242ffd700 1 -- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62440ad750 msgr2=0x7f624414db50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.844+0000 7f6242ffd700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62440ad750 0x7f624414db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.844+0000 7f6242ffd700 1 -- 172.21.15.100:0/4110074397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340094f0 con 0x7f62440b0d90 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.844+0000 7f6243fff700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6244156d80 0x7f624413fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.844+0000 7f6242ffd700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f624414a870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6234009290 tx=0x7f6234017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f622b7fe700 1 -- 172.21.15.100:0/4110074397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62340119e0 con 0x7f62440b0d90 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62440cad70 con 0x7f62440b0d90 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624414e230 con 0x7f62440b0d90 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f622b7fe700 1 -- 172.21.15.100:0/4110074397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6234011b40 con 0x7f62440b0d90 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f622b7fe700 1 -- 172.21.15.100:0/4110074397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623402d3f0 con 0x7f62440b0d90 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62240052b0 con 0x7f62440b0d90 2024-08-17T02:22:15.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f622b7fe700 1 -- 172.21.15.100:0/4110074397 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f623402d820 con 0x7f62440b0d90 2024-08-17T02:22:15.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f622b7fe700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6230060930 0x7f6230062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:15.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f622b7fe700 1 -- 172.21.15.100:0/4110074397 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6234086c80 con 0x7f62440b0d90 2024-08-17T02:22:15.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.852+0000 7f62437fe700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6230060930 0x7f6230062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:15.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.856+0000 7f62437fe700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6230060930 0x7f6230062de0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6238009b10 tx=0x7f62380091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:15.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:15.860+0000 7f622b7fe700 1 -- 172.21.15.100:0/4110074397 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6234057150 con 0x7f62440b0d90 2024-08-17T02:22:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.056+0000 7f75a158b700 1 -- 172.21.15.100:0/1334938149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7578005830 con 0x7f759c13bf10 2024-08-17T02:22:16.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.056+0000 7f758affd700 1 -- 172.21.15.100:0/1334938149 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7594018090 con 0x7f759c13bf10 2024-08-17T02:22:16.108 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 -- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7580060860 msgr2=0x7f7580062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7580060860 0x7f7580062d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f758c006fd0 tx=0x7f758c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 -- 172.21.15.100:0/1334938149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c13bf10 msgr2=0x7f759c0ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c13bf10 0x7f759c0ca050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f75940076d0 tx=0x7f759401a000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 -- 172.21.15.100:0/1334938149 shutdown_connections 2024-08-17T02:22:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7580060860 0x7f7580062d10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f759c13bf10 0x7f759c0ca050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f759c13b630 0x7f759c0c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 --2- 172.21.15.100:0/1334938149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f759c0b1950 0x7f759c16d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 -- 172.21.15.100:0/1334938149 >> 172.21.15.100:0/1334938149 conn(0x7f759c0678d0 msgr2=0x7f759c069e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 -- 172.21.15.100:0/1334938149 shutdown_connections 2024-08-17T02:22:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.060+0000 7f7588ff9700 1 -- 172.21.15.100:0/1334938149 wait complete. 2024-08-17T02:22:16.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-08-17T02:22:16.111 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.108+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7b300aee30 con 0x7f7b3015d910 2024-08-17T02:22:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.108+0000 7f7b1effd700 1 -- 172.21.15.100:0/3125434479 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b300aee30 con 0x7f7b3015d910 2024-08-17T02:22:16.112 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7b140609b0 msgr2=0x7f7b14062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7b140609b0 0x7f7b14062e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b200099b0 tx=0x7f7b20016000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3015d910 msgr2=0x7f7b30169d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3015d910 0x7f7b30169d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400d8c0 tx=0x7f7b24008040 comp rx=0 tx=0).stop 2024-08-17T02:22:16.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 shutdown_connections 2024-08-17T02:22:16.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7b140609b0 0x7f7b14062e60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3015d910 0x7f7b30169d10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b30158fb0 0x7f7b301697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 --2- 172.21.15.100:0/3125434479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b3006e620 0x7f7b300c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 >> 172.21.15.100:0/3125434479 conn(0x7f7b300678d0 msgr2=0x7f7b300aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 shutdown_connections 2024-08-17T02:22:16.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.112+0000 7f7b34867700 1 -- 172.21.15.100:0/3125434479 wait complete. 2024-08-17T02:22:16.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.120+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcb14151520 con 0x7fcb140b14a0 2024-08-17T02:22:16.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb0a7fc700 1 -- 172.21.15.100:0/2430213887 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb10017030 con 0x7fcb140b14a0 2024-08-17T02:22:16.125 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:16.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5080005830 con 0x7f50a00b1630 2024-08-17T02:22:16.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7f50957fa700 1 -- 172.21.15.100:0/46645999 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5098050ec0 con 0x7f50a00b1630 2024-08-17T02:22:16.126 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:16.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6224005830 con 0x7f62440b0d90 2024-08-17T02:22:16.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcaf8060930 msgr2=0x7fcaf8062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcaf8060930 0x7fcaf8062de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb040098e0 tx=0x7fcb04017000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 msgr2=0x7fcb140a7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140a7c10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c790 tx=0x7fcb10007df0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7f622b7fe700 1 -- 172.21.15.100:0/4110074397 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f623405a770 con 0x7f62440b0d90 2024-08-17T02:22:16.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 shutdown_connections 2024-08-17T02:22:16.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcaf8060930 0x7fcaf8062de0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb14154900 0x7fcb140a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb140cb390 0x7fcb140b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 --2- 172.21.15.100:0/2430213887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb140b14a0 0x7fcb140a7c10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 >> 172.21.15.100:0/2430213887 conn(0x7fcb140678d0 msgr2=0x7fcb1406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.129 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:16.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-08-17T02:22:16.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 shutdown_connections 2024-08-17T02:22:16.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.124+0000 7fcb1b4c2700 1 -- 172.21.15.100:0/2430213887 wait complete. 2024-08-17T02:22:16.131 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5084060860 msgr2=0x7f5084062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5084060860 0x7f5084062d10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f509c0099b0 tx=0x7f509c016000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a00b1630 msgr2=0x7f50a0160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a00b1630 0x7f50a0160270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f509800cb90 tx=0x7f5098001820 comp rx=0 tx=0).stop 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 shutdown_connections 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5084060860 0x7f5084062d10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a00b1630 0x7f50a0160270 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50a00b0d20 0x7f50a015fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 --2- 172.21.15.100:0/46645999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a006e450 0x7f50a016e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 >> 172.21.15.100:0/46645999 conn(0x7f50a00678d0 msgr2=0x7f50a0068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 shutdown_connections 2024-08-17T02:22:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f50a63cc700 1 -- 172.21.15.100:0/46645999 wait complete. 2024-08-17T02:22:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6230060930 msgr2=0x7f6230062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6230060930 0x7f6230062de0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6238009b10 tx=0x7f62380091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 msgr2=0x7f624414a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f624414a870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6234009290 tx=0x7f6234017000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 shutdown_connections 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6230060930 0x7f6230062de0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6244156d80 0x7f624413fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62440b0d90 0x7f624414a870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 --2- 172.21.15.100:0/4110074397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62440ad750 0x7f624414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 >> 172.21.15.100:0/4110074397 conn(0x7f62440678d0 msgr2=0x7f62440aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 shutdown_connections 2024-08-17T02:22:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.128+0000 7f6248f0b700 1 -- 172.21.15.100:0/4110074397 wait complete. 2024-08-17T02:22:16.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-08-17T02:22:16.141 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:16.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-08-17T02:22:16.142 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:16.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-08-17T02:22:16.144 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2ecc005160 con 0x7f2ef406f2b0 2024-08-17T02:22:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb66c0aaa70 con 0x7fb66c0a9d00 2024-08-17T02:22:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2ef0ff9700 1 -- 172.21.15.100:0/3369419719 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2eec051270 con 0x7f2ef406f2b0 2024-08-17T02:22:16.146 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb65a7fc700 1 -- 172.21.15.100:0/853515707 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb66c0aaa70 con 0x7fb66c0a9d00 2024-08-17T02:22:16.147 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:16.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2edc060a10 msgr2=0x7f2edc062ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2edc060a10 0x7f2edc062ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40705d0 tx=0x7f2ee8017000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef406f2b0 msgr2=0x7f2ef41484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef406f2b0 0x7f2ef41484d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00e190 tx=0x7f2eec009920 comp rx=0 tx=0).stop 2024-08-17T02:22:16.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 shutdown_connections 2024-08-17T02:22:16.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2edc060a10 0x7f2edc062ec0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ef414fc20 0x7f2ef416ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef414f850 0x7f2ef40afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 --2- 172.21.15.100:0/3369419719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef406f2b0 0x7f2ef41484d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 >> 172.21.15.100:0/3369419719 conn(0x7f2ef4068300 msgr2=0x7f2ef4156f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 shutdown_connections 2024-08-17T02:22:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f2efb110700 1 -- 172.21.15.100:0/3369419719 wait complete. 2024-08-17T02:22:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb654066420 msgr2=0x7fb6540688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb654066420 0x7fb6540688d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009cd0 tx=0x7fb65c008040 comp rx=0 tx=0).stop 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb66c0a9d00 msgr2=0x7fb66c1418c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb66c0a9d00 0x7fb66c1418c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb664009e00 tx=0x7fb66400b040 comp rx=0 tx=0).stop 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 shutdown_connections 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb654066420 0x7fb6540688d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb66c0aa170 0x7fb66c0af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb66c0a9d00 0x7fb66c1418c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 --2- 172.21.15.100:0/853515707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6540051c0 0x7fb66c168ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 >> 172.21.15.100:0/853515707 conn(0x7fb66c06db30 msgr2=0x7fb66c0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7eb80645c0 con 0x7f7eb816b050 2024-08-17T02:22:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 shutdown_connections 2024-08-17T02:22:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.144+0000 7fb670bc1700 1 -- 172.21.15.100:0/853515707 wait complete. 2024-08-17T02:22:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.148+0000 7f7eb4ff9700 1 -- 172.21.15.100:0/295322474 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ea8015030 con 0x7f7eb816b050 2024-08-17T02:22:16.151 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.148+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7ea4060890 msgr2=0x7f7ea4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.148+0000 7f7ebf071700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7ea4060890 0x7f7ea4062d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000a0e0 tx=0x7f7eb0002180 comp rx=0 tx=0).stop 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.148+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb816b050 msgr2=0x7f7eb80a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.148+0000 7f7ebf071700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb816b050 0x7f7eb80a9c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8016000 tx=0x7f7ea8019000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.152+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 shutdown_connections 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.152+0000 7f7ebf071700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7ea4060890 0x7f7ea4062d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.152+0000 7f7ebf071700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7eb816b050 0x7f7eb80a9c60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.152+0000 7f7ebf071700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7eb816b8e0 0x7f7eb816ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.152+0000 7f7ebf071700 1 --2- 172.21.15.100:0/295322474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea0004f00 0x7f7eb8157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.152+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 >> 172.21.15.100:0/295322474 conn(0x7f7eb8067810 msgr2=0x7f7eb80cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.152+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 shutdown_connections 2024-08-17T02:22:16.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.152+0000 7f7ebf071700 1 -- 172.21.15.100:0/295322474 wait complete. 2024-08-17T02:22:16.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-08-17T02:22:16.159 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:16.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2024-08-17T02:22:16.164 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:16.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-08-17T02:22:16.166 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:16.166 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-17T02:22:16.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.308+0000 7f1113278700 1 Processor -- start 2024-08-17T02:22:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.308+0000 7f1113278700 1 -- start start 2024-08-17T02:22:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.308+0000 7f1113278700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f110c15d540 0x7f110c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.308+0000 7f1113278700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c0a9430 0x7f110c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.308+0000 7f1113278700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c0b0270 con 0x7f110c06e830 2024-08-17T02:22:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.308+0000 7f1113278700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c0af660 con 0x7f110c15d540 2024-08-17T02:22:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.308+0000 7f1113278700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0af940 con 0x7f110c0a9430 2024-08-17T02:22:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111014700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f110c15d540 0x7f110c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111014700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f110c15d540 0x7f110c0af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52494/0 (socket says 172.21.15.100:52494) 2024-08-17T02:22:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111014700 1 -- 172.21.15.100:0/2807619147 learned_addr learned my addr 172.21.15.100:0/2807619147 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111815700 1 --2- 172.21.15.100:0/2807619147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c0a9430 0x7f110c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111014700 1 -- 172.21.15.100:0/2807619147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c0a9430 msgr2=0x7f110c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111014700 1 --2- 172.21.15.100:0/2807619147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c0a9430 0x7f110c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111014700 1 -- 172.21.15.100:0/2807619147 >> v1:172.21.15.123:6789/0 conn(0x7f110c06e830 legacy=0x7f110c0a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111014700 1 -- 172.21.15.100:0/2807619147 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c0afc20 con 0x7f110c15d540 2024-08-17T02:22:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f110bfff700 1 -- 172.21.15.100:0/2807619147 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288714644 0 0) 0x7f110c0b0270 con 0x7f110c06e830 2024-08-17T02:22:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1111014700 1 --2- 172.21.15.100:0/2807619147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c15d540 0x7f110c0af030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009f20 tx=0x7f10fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deee7636776f846a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f110bfff700 1 -- 172.21.15.100:0/2807619147 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00e070 con 0x7f110c15d540 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f110bfff700 1 -- 172.21.15.100:0/2807619147 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc00b9d0 con 0x7f110c15d540 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f110bfff700 1 -- 172.21.15.100:0/2807619147 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0092e0 con 0x7f110c15d540 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 -- 172.21.15.100:0/2807619147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c15d540 msgr2=0x7f110c0af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 --2- 172.21.15.100:0/2807619147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c15d540 0x7f110c0af030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009f20 tx=0x7f10fc009840 comp rx=0 tx=0).stop 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 -- 172.21.15.100:0/2807619147 shutdown_connections 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 --2- 172.21.15.100:0/2807619147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c0a9430 0x7f110c0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 --2- 172.21.15.100:0/2807619147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c15d540 0x7f110c0af030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 -- 172.21.15.100:0/2807619147 >> 172.21.15.100:0/2807619147 conn(0x7f110c066e00 msgr2=0x7f110c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 -- 172.21.15.100:0/2807619147 shutdown_connections 2024-08-17T02:22:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 -- 172.21.15.100:0/2807619147 wait complete. 2024-08-17T02:22:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 Processor -- start 2024-08-17T02:22:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.312+0000 7f1113278700 1 -- start start 2024-08-17T02:22:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1113278700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c06e830 0x7f110c0980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1113278700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0a9430 0x7f110c098600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1113278700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f110c15d540 0x7f110c0ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1113278700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c097020 con 0x7f110c0a9430 2024-08-17T02:22:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1113278700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c097300 con 0x7f110c15d540 2024-08-17T02:22:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1113278700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0975e0 con 0x7f110c06e830 2024-08-17T02:22:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1110813700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0a9430 0x7f110c098600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1111014700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c06e830 0x7f110c0980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1111014700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c06e830 0x7f110c0980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:54232/0 (socket says 172.21.15.100:54232) 2024-08-17T02:22:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1111014700 1 -- 172.21.15.100:0/172215747 learned_addr learned my addr 172.21.15.100:0/172215747 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1111815700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f110c15d540 0x7f110c0ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1110813700 1 -- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c06e830 msgr2=0x7f110c0980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1110813700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c06e830 0x7f110c0980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1110813700 1 -- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f110c15d540 msgr2=0x7f110c0ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1110813700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f110c15d540 0x7f110c0ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1110813700 1 -- 172.21.15.100:0/172215747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc009ae0 con 0x7f110c0a9430 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1111815700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f110c15d540 0x7f110c0ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1111014700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c06e830 0x7f110c0980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1110813700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0a9430 0x7f110c098600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11000011e0 tx=0x7f1100007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f11097fa700 1 -- 172.21.15.100:0/172215747 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100011590 con 0x7f110c0a9430 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f11097fa700 1 -- 172.21.15.100:0/172215747 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1100011710 con 0x7f110c0a9430 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c0978c0 con 0x7f110c0a9430 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f11097fa700 1 -- 172.21.15.100:0/172215747 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100011ad0 con 0x7f110c0a9430 2024-08-17T02:22:16.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.316+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c158e20 con 0x7f110c0a9430 2024-08-17T02:22:16.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.320+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c0645c0 con 0x7f110c0a9430 2024-08-17T02:22:16.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.320+0000 7f11097fa700 1 -- 172.21.15.100:0/172215747 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f110001b3b0 con 0x7f110c0a9430 2024-08-17T02:22:16.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.320+0000 7f11097fa700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f10f8060860 0x7f10f8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.320+0000 7f11097fa700 1 -- 172.21.15.100:0/172215747 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f110007c050 con 0x7f110c0a9430 2024-08-17T02:22:16.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.320+0000 7f1111014700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f10f8060860 0x7f10f8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.320+0000 7f1111014700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f10f8060860 0x7f10f8062d10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009880 tx=0x7f10fc018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.324+0000 7f11097fa700 1 -- 172.21.15.100:0/172215747 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f110004c400 con 0x7f110c0a9430 2024-08-17T02:22:16.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.476+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f110c06f360 con 0x7f10f8060860 2024-08-17T02:22:16.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.492+0000 7f11097fa700 1 -- 172.21.15.100:0/172215747 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264365 (secure 0 0 0) 0x7f110c06f360 con 0x7f10f8060860 2024-08-17T02:22:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f10f8060860 msgr2=0x7f10f8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f10f8060860 0x7f10f8062d10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009880 tx=0x7f10fc018000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0a9430 msgr2=0x7f110c098600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0a9430 0x7f110c098600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11000011e0 tx=0x7f1100007d30 comp rx=0 tx=0).stop 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 shutdown_connections 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f10f8060860 0x7f10f8062d10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f110c15d540 0x7f110c0ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0a9430 0x7f110c098600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 --2- 172.21.15.100:0/172215747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f110c06e830 0x7f110c0980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 >> 172.21.15.100:0/172215747 conn(0x7f110c066e00 msgr2=0x7f110c15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 shutdown_connections 2024-08-17T02:22:16.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.496+0000 7f1113278700 1 -- 172.21.15.100:0/172215747 wait complete. 2024-08-17T02:22:16.502 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:16.513 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-08-17T02:22:15.224721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382048,"kb_used_data":2984,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592672,"statfs":{"total":773094113280,"available":770654896128,"internally_reserved":0,"allocated":3055616,"data_stored":1812404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003598"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143110+0000","last_change":"2024-08-17T02:22:05.137118+0000","last_active":"2024-08-17T02:22:05.143110+0000","last_peered":"2024-08-17T02:22:05.143110+0000","last_clean":"2024-08-17T02:22:05.143110+0000","last_became_active":"2024-08-17T02:22:05.136784+0000","last_became_peered":"2024-08-17T02:22:05.136784+0000","last_unstale":"2024-08-17T02:22:05.143110+0000","last_undegraded":"2024-08-17T02:22:05.143110+0000","last_fullsized":"2024-08-17T02:22:05.143110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:04:25.260837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133391+0000","last_change":"2024-08-17T02:22:05.129636+0000","last_active":"2024-08-17T02:22:05.133391+0000","last_peered":"2024-08-17T02:22:05.133391+0000","last_clean":"2024-08-17T02:22:05.133391+0000","last_became_active":"2024-08-17T02:22:05.129342+0000","last_became_peered":"2024-08-17T02:22:05.129342+0000","last_unstale":"2024-08-17T02:22:05.133391+0000","last_undegraded":"2024-08-17T02:22:05.133391+0000","last_fullsized":"2024-08-17T02:22:05.133391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.131713+0000","last_change":"2024-08-17T02:22:05.128097+0000","last_active":"2024-08-17T02:22:05.131713+0000","last_peered":"2024-08-17T02:22:05.131713+0000","last_clean":"2024-08-17T02:22:05.131713+0000","last_became_active":"2024-08-17T02:22:05.127786+0000","last_became_peered":"2024-08-17T02:22:05.127786+0000","last_unstale":"2024-08-17T02:22:05.131713+0000","last_undegraded":"2024-08-17T02:22:05.131713+0000","last_fullsized":"2024-08-17T02:22:05.131713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:47.605286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142739+0000","last_change":"2024-08-17T02:22:05.138911+0000","last_active":"2024-08-17T02:22:05.142739+0000","last_peered":"2024-08-17T02:22:05.142739+0000","last_clean":"2024-08-17T02:22:05.142739+0000","last_became_active":"2024-08-17T02:22:05.134886+0000","last_became_peered":"2024-08-17T02:22:05.134886+0000","last_unstale":"2024-08-17T02:22:05.142739+0000","last_undegraded":"2024-08-17T02:22:05.142739+0000","last_fullsized":"2024-08-17T02:22:05.142739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:04:25.260837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143121+0000","last_change":"2024-08-17T02:22:05.137212+0000","last_active":"2024-08-17T02:22:05.143121+0000","last_peered":"2024-08-17T02:22:05.143121+0000","last_clean":"2024-08-17T02:22:05.143121+0000","last_became_active":"2024-08-17T02:22:05.136566+0000","last_became_peered":"2024-08-17T02:22:05.136566+0000","last_unstale":"2024-08-17T02:22:05.143121+0000","last_undegraded":"2024-08-17T02:22:05.143121+0000","last_fullsized":"2024-08-17T02:22:05.143121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:49.230407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141537+0000","last_change":"2024-08-17T02:22:05.130974+0000","last_active":"2024-08-17T02:22:05.141537+0000","last_peered":"2024-08-17T02:22:05.141537+0000","last_clean":"2024-08-17T02:22:05.141537+0000","last_became_active":"2024-08-17T02:22:05.130696+0000","last_became_peered":"2024-08-17T02:22:05.130696+0000","last_unstale":"2024-08-17T02:22:05.141537+0000","last_undegraded":"2024-08-17T02:22:05.141537+0000","last_fullsized":"2024-08-17T02:22:05.141537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:49.230407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143027+0000","last_change":"2024-08-17T02:22:05.137564+0000","last_active":"2024-08-17T02:22:05.143027+0000","last_peered":"2024-08-17T02:22:05.143027+0000","last_clean":"2024-08-17T02:22:05.143027+0000","last_became_active":"2024-08-17T02:22:05.137060+0000","last_became_peered":"2024-08-17T02:22:05.137060+0000","last_unstale":"2024-08-17T02:22:05.143027+0000","last_undegraded":"2024-08-17T02:22:05.143027+0000","last_fullsized":"2024-08-17T02:22:05.143027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:48:26.336166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146835+0000","last_change":"2024-08-17T02:22:05.143886+0000","last_active":"2024-08-17T02:22:05.146835+0000","last_peered":"2024-08-17T02:22:05.146835+0000","last_clean":"2024-08-17T02:22:05.146835+0000","last_became_active":"2024-08-17T02:22:05.143537+0000","last_became_peered":"2024-08-17T02:22:05.143537+0000","last_unstale":"2024-08-17T02:22:05.146835+0000","last_undegraded":"2024-08-17T02:22:05.146835+0000","last_fullsized":"2024-08-17T02:22:05.146835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:58:32.777645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146777+0000","last_change":"2024-08-17T02:22:05.142931+0000","last_active":"2024-08-17T02:22:05.146777+0000","last_peered":"2024-08-17T02:22:05.146777+0000","last_clean":"2024-08-17T02:22:05.146777+0000","last_became_active":"2024-08-17T02:22:05.142076+0000","last_became_peered":"2024-08-17T02:22:05.142076+0000","last_unstale":"2024-08-17T02:22:05.146777+0000","last_undegraded":"2024-08-17T02:22:05.146777+0000","last_fullsized":"2024-08-17T02:22:05.146777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:54:13.258567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133864+0000","last_change":"2024-08-17T02:22:05.131490+0000","last_active":"2024-08-17T02:22:05.133864+0000","last_peered":"2024-08-17T02:22:05.133864+0000","last_clean":"2024-08-17T02:22:05.133864+0000","last_became_active":"2024-08-17T02:22:05.131184+0000","last_became_peered":"2024-08-17T02:22:05.131184+0000","last_unstale":"2024-08-17T02:22:05.133864+0000","last_undegraded":"2024-08-17T02:22:05.133864+0000","last_fullsized":"2024-08-17T02:22:05.133864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134444+0000","last_change":"2024-08-17T02:22:05.131423+0000","last_active":"2024-08-17T02:22:05.134444+0000","last_peered":"2024-08-17T02:22:05.134444+0000","last_clean":"2024-08-17T02:22:05.134444+0000","last_became_active":"2024-08-17T02:22:05.131081+0000","last_became_peered":"2024-08-17T02:22:05.131081+0000","last_unstale":"2024-08-17T02:22:05.134444+0000","last_undegraded":"2024-08-17T02:22:05.134444+0000","last_fullsized":"2024-08-17T02:22:05.134444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133327+0000","last_change":"2024-08-17T02:22:05.129487+0000","last_active":"2024-08-17T02:22:05.133327+0000","last_peered":"2024-08-17T02:22:05.133327+0000","last_clean":"2024-08-17T02:22:05.133327+0000","last_became_active":"2024-08-17T02:22:05.129222+0000","last_became_peered":"2024-08-17T02:22:05.129222+0000","last_unstale":"2024-08-17T02:22:05.133327+0000","last_undegraded":"2024-08-17T02:22:05.133327+0000","last_fullsized":"2024-08-17T02:22:05.133327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:02.706489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138694+0000","last_change":"2024-08-17T02:22:05.134693+0000","last_active":"2024-08-17T02:22:05.138694+0000","last_peered":"2024-08-17T02:22:05.138694+0000","last_clean":"2024-08-17T02:22:05.138694+0000","last_became_active":"2024-08-17T02:22:05.134300+0000","last_became_peered":"2024-08-17T02:22:05.134300+0000","last_unstale":"2024-08-17T02:22:05.138694+0000","last_undegraded":"2024-08-17T02:22:05.138694+0000","last_fullsized":"2024-08-17T02:22:05.138694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:37.064590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.131330+0000","last_change":"2024-08-17T02:22:05.128086+0000","last_active":"2024-08-17T02:22:05.131330+0000","last_peered":"2024-08-17T02:22:05.131330+0000","last_clean":"2024-08-17T02:22:05.131330+0000","last_became_active":"2024-08-17T02:22:05.127750+0000","last_became_peered":"2024-08-17T02:22:05.127750+0000","last_unstale":"2024-08-17T02:22:05.131330+0000","last_undegraded":"2024-08-17T02:22:05.131330+0000","last_fullsized":"2024-08-17T02:22:05.131330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:01:27.557725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.123798+0000","last_change":"2024-08-17T02:22:05.123028+0000","last_active":"2024-08-17T02:22:05.123798+0000","last_peered":"2024-08-17T02:22:05.123798+0000","last_clean":"2024-08-17T02:22:05.123798+0000","last_became_active":"2024-08-17T02:22:05.122711+0000","last_became_peered":"2024-08-17T02:22:05.122711+0000","last_unstale":"2024-08-17T02:22:05.123798+0000","last_undegraded":"2024-08-17T02:22:05.123798+0000","last_fullsized":"2024-08-17T02:22:05.123798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141452+0000","last_change":"2024-08-17T02:22:05.130380+0000","last_active":"2024-08-17T02:22:05.141452+0000","last_peered":"2024-08-17T02:22:05.141452+0000","last_clean":"2024-08-17T02:22:05.141452+0000","last_became_active":"2024-08-17T02:22:05.130077+0000","last_became_peered":"2024-08-17T02:22:05.130077+0000","last_unstale":"2024-08-17T02:22:05.141452+0000","last_undegraded":"2024-08-17T02:22:05.141452+0000","last_fullsized":"2024-08-17T02:22:05.141452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:27:57.075296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142733+0000","last_change":"2024-08-17T02:22:05.139113+0000","last_active":"2024-08-17T02:22:05.142733+0000","last_peered":"2024-08-17T02:22:05.142733+0000","last_clean":"2024-08-17T02:22:05.142733+0000","last_became_active":"2024-08-17T02:22:05.138752+0000","last_became_peered":"2024-08-17T02:22:05.138752+0000","last_unstale":"2024-08-17T02:22:05.142733+0000","last_undegraded":"2024-08-17T02:22:05.142733+0000","last_fullsized":"2024-08-17T02:22:05.142733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:21:52.626562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.129913+0000","last_change":"2024-08-17T02:22:05.125930+0000","last_active":"2024-08-17T02:22:05.129913+0000","last_peered":"2024-08-17T02:22:05.129913+0000","last_clean":"2024-08-17T02:22:05.129913+0000","last_became_active":"2024-08-17T02:22:05.125640+0000","last_became_peered":"2024-08-17T02:22:05.125640+0000","last_unstale":"2024-08-17T02:22:05.129913+0000","last_undegraded":"2024-08-17T02:22:05.129913+0000","last_fullsized":"2024-08-17T02:22:05.129913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:59:54.522675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142959+0000","last_change":"2024-08-17T02:22:05.137818+0000","last_active":"2024-08-17T02:22:05.142959+0000","last_peered":"2024-08-17T02:22:05.142959+0000","last_clean":"2024-08-17T02:22:05.142959+0000","last_became_active":"2024-08-17T02:22:05.137531+0000","last_became_peered":"2024-08-17T02:22:05.137531+0000","last_unstale":"2024-08-17T02:22:05.142959+0000","last_undegraded":"2024-08-17T02:22:05.142959+0000","last_fullsized":"2024-08-17T02:22:05.142959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:44:40.942386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.137413+0000","last_change":"2024-08-17T02:22:05.135163+0000","last_active":"2024-08-17T02:22:05.137413+0000","last_peered":"2024-08-17T02:22:05.137413+0000","last_clean":"2024-08-17T02:22:05.137413+0000","last_became_active":"2024-08-17T02:22:05.134647+0000","last_became_peered":"2024-08-17T02:22:05.134647+0000","last_unstale":"2024-08-17T02:22:05.137413+0000","last_undegraded":"2024-08-17T02:22:05.137413+0000","last_fullsized":"2024-08-17T02:22:05.137413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:21.690535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144205+0000","last_change":"2024-08-17T02:22:05.137440+0000","last_active":"2024-08-17T02:22:05.144205+0000","last_peered":"2024-08-17T02:22:05.144205+0000","last_clean":"2024-08-17T02:22:05.144205+0000","last_became_active":"2024-08-17T02:22:05.137081+0000","last_became_peered":"2024-08-17T02:22:05.137081+0000","last_unstale":"2024-08-17T02:22:05.144205+0000","last_undegraded":"2024-08-17T02:22:05.144205+0000","last_fullsized":"2024-08-17T02:22:05.144205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:34:43.102929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142978+0000","last_change":"2024-08-17T02:22:05.139220+0000","last_active":"2024-08-17T02:22:05.142978+0000","last_peered":"2024-08-17T02:22:05.142978+0000","last_clean":"2024-08-17T02:22:05.142978+0000","last_became_active":"2024-08-17T02:22:05.138838+0000","last_became_peered":"2024-08-17T02:22:05.138838+0000","last_unstale":"2024-08-17T02:22:05.142978+0000","last_undegraded":"2024-08-17T02:22:05.142978+0000","last_fullsized":"2024-08-17T02:22:05.142978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:47:26.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135504+0000","last_change":"2024-08-17T02:22:05.132463+0000","last_active":"2024-08-17T02:22:05.135504+0000","last_peered":"2024-08-17T02:22:05.135504+0000","last_clean":"2024-08-17T02:22:05.135504+0000","last_became_active":"2024-08-17T02:22:05.131741+0000","last_became_peered":"2024-08-17T02:22:05.131741+0000","last_unstale":"2024-08-17T02:22:05.135504+0000","last_undegraded":"2024-08-17T02:22:05.135504+0000","last_fullsized":"2024-08-17T02:22:05.135504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142777+0000","last_change":"2024-08-17T02:22:05.140510+0000","last_active":"2024-08-17T02:22:05.142777+0000","last_peered":"2024-08-17T02:22:05.142777+0000","last_clean":"2024-08-17T02:22:05.142777+0000","last_became_active":"2024-08-17T02:22:05.139145+0000","last_became_peered":"2024-08-17T02:22:05.139145+0000","last_unstale":"2024-08-17T02:22:05.142777+0000","last_undegraded":"2024-08-17T02:22:05.142777+0000","last_fullsized":"2024-08-17T02:22:05.142777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:58:53.092915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135524+0000","last_change":"2024-08-17T02:22:05.133736+0000","last_active":"2024-08-17T02:22:05.135524+0000","last_peered":"2024-08-17T02:22:05.135524+0000","last_clean":"2024-08-17T02:22:05.135524+0000","last_became_active":"2024-08-17T02:22:05.132607+0000","last_became_peered":"2024-08-17T02:22:05.132607+0000","last_unstale":"2024-08-17T02:22:05.135524+0000","last_undegraded":"2024-08-17T02:22:05.135524+0000","last_fullsized":"2024-08-17T02:22:05.135524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141845+0000","last_change":"2024-08-17T02:22:05.137747+0000","last_active":"2024-08-17T02:22:05.141845+0000","last_peered":"2024-08-17T02:22:05.141845+0000","last_clean":"2024-08-17T02:22:05.141845+0000","last_became_active":"2024-08-17T02:22:05.137464+0000","last_became_peered":"2024-08-17T02:22:05.137464+0000","last_unstale":"2024-08-17T02:22:05.141845+0000","last_undegraded":"2024-08-17T02:22:05.141845+0000","last_fullsized":"2024-08-17T02:22:05.141845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138959+0000","last_change":"2024-08-17T02:22:05.133235+0000","last_active":"2024-08-17T02:22:05.138959+0000","last_peered":"2024-08-17T02:22:05.138959+0000","last_clean":"2024-08-17T02:22:05.138959+0000","last_became_active":"2024-08-17T02:22:05.132962+0000","last_became_peered":"2024-08-17T02:22:05.132962+0000","last_unstale":"2024-08-17T02:22:05.138959+0000","last_undegraded":"2024-08-17T02:22:05.138959+0000","last_fullsized":"2024-08-17T02:22:05.138959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135434+0000","last_change":"2024-08-17T02:22:05.132315+0000","last_active":"2024-08-17T02:22:05.135434+0000","last_peered":"2024-08-17T02:22:05.135434+0000","last_clean":"2024-08-17T02:22:05.135434+0000","last_became_active":"2024-08-17T02:22:05.131822+0000","last_became_peered":"2024-08-17T02:22:05.131822+0000","last_unstale":"2024-08-17T02:22:05.135434+0000","last_undegraded":"2024-08-17T02:22:05.135434+0000","last_fullsized":"2024-08-17T02:22:05.135434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139793+0000","last_change":"2024-08-17T02:22:05.136258+0000","last_active":"2024-08-17T02:22:05.139793+0000","last_peered":"2024-08-17T02:22:05.139793+0000","last_clean":"2024-08-17T02:22:05.139793+0000","last_became_active":"2024-08-17T02:22:05.135573+0000","last_became_peered":"2024-08-17T02:22:05.135573+0000","last_unstale":"2024-08-17T02:22:05.139793+0000","last_undegraded":"2024-08-17T02:22:05.139793+0000","last_fullsized":"2024-08-17T02:22:05.139793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:36:55.030030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142962+0000","last_change":"2024-08-17T02:22:05.139131+0000","last_active":"2024-08-17T02:22:05.142962+0000","last_peered":"2024-08-17T02:22:05.142962+0000","last_clean":"2024-08-17T02:22:05.142962+0000","last_became_active":"2024-08-17T02:22:05.138484+0000","last_became_peered":"2024-08-17T02:22:05.138484+0000","last_unstale":"2024-08-17T02:22:05.142962+0000","last_undegraded":"2024-08-17T02:22:05.142962+0000","last_fullsized":"2024-08-17T02:22:05.142962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:49:31.875303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134588+0000","last_change":"2024-08-17T02:22:05.128466+0000","last_active":"2024-08-17T02:22:05.134588+0000","last_peered":"2024-08-17T02:22:05.134588+0000","last_clean":"2024-08-17T02:22:05.134588+0000","last_became_active":"2024-08-17T02:22:05.128215+0000","last_became_peered":"2024-08-17T02:22:05.128215+0000","last_unstale":"2024-08-17T02:22:05.134588+0000","last_undegraded":"2024-08-17T02:22:05.134588+0000","last_fullsized":"2024-08-17T02:22:05.134588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:49.930124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133891+0000","last_change":"2024-08-17T02:22:05.131735+0000","last_active":"2024-08-17T02:22:05.133891+0000","last_peered":"2024-08-17T02:22:05.133891+0000","last_clean":"2024-08-17T02:22:05.133891+0000","last_became_active":"2024-08-17T02:22:05.131343+0000","last_became_peered":"2024-08-17T02:22:05.131343+0000","last_unstale":"2024-08-17T02:22:05.133891+0000","last_undegraded":"2024-08-17T02:22:05.133891+0000","last_fullsized":"2024-08-17T02:22:05.133891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:41:42.553817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.128947+0000","last_change":"2024-08-17T02:22:05.124427+0000","last_active":"2024-08-17T02:22:05.128947+0000","last_peered":"2024-08-17T02:22:05.128947+0000","last_clean":"2024-08-17T02:22:05.128947+0000","last_became_active":"2024-08-17T02:22:05.124149+0000","last_became_peered":"2024-08-17T02:22:05.124149+0000","last_unstale":"2024-08-17T02:22:05.128947+0000","last_undegraded":"2024-08-17T02:22:05.128947+0000","last_fullsized":"2024-08-17T02:22:05.128947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:57.001696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.140231+0000","last_change":"2024-08-17T02:22:04.140654+0000","last_active":"2024-08-17T02:22:05.140231+0000","last_peered":"2024-08-17T02:22:05.140231+0000","last_clean":"2024-08-17T02:22:05.140231+0000","last_became_active":"2024-08-17T02:22:04.140316+0000","last_became_peered":"2024-08-17T02:22:04.140316+0000","last_unstale":"2024-08-17T02:22:05.140231+0000","last_undegraded":"2024-08-17T02:22:05.140231+0000","last_fullsized":"2024-08-17T02:22:05.140231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136973+0000","last_change":"2024-08-17T02:22:05.130524+0000","last_active":"2024-08-17T02:22:05.136973+0000","last_peered":"2024-08-17T02:22:05.136973+0000","last_clean":"2024-08-17T02:22:05.136973+0000","last_became_active":"2024-08-17T02:22:05.130188+0000","last_became_peered":"2024-08-17T02:22:05.130188+0000","last_unstale":"2024-08-17T02:22:05.136973+0000","last_undegraded":"2024-08-17T02:22:05.136973+0000","last_fullsized":"2024-08-17T02:22:05.136973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:11:14.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134335+0000","last_change":"2024-08-17T02:22:04.123306+0000","last_active":"2024-08-17T02:22:05.134335+0000","last_peered":"2024-08-17T02:22:05.134335+0000","last_clean":"2024-08-17T02:22:05.134335+0000","last_became_active":"2024-08-17T02:22:04.122968+0000","last_became_peered":"2024-08-17T02:22:04.122968+0000","last_unstale":"2024-08-17T02:22:05.134335+0000","last_undegraded":"2024-08-17T02:22:05.134335+0000","last_fullsized":"2024-08-17T02:22:05.134335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:26:59.050552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.124555+0000","last_change":"2024-08-17T02:22:04.124101+0000","last_active":"2024-08-17T02:22:05.124555+0000","last_peered":"2024-08-17T02:22:05.124555+0000","last_clean":"2024-08-17T02:22:05.124555+0000","last_became_active":"2024-08-17T02:22:04.123756+0000","last_became_peered":"2024-08-17T02:22:04.123756+0000","last_unstale":"2024-08-17T02:22:05.124555+0000","last_undegraded":"2024-08-17T02:22:05.124555+0000","last_fullsized":"2024-08-17T02:22:05.124555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135564+0000","last_change":"2024-08-17T02:22:05.133677+0000","last_active":"2024-08-17T02:22:05.135564+0000","last_peered":"2024-08-17T02:22:05.135564+0000","last_clean":"2024-08-17T02:22:05.135564+0000","last_became_active":"2024-08-17T02:22:05.132419+0000","last_became_peered":"2024-08-17T02:22:05.132419+0000","last_unstale":"2024-08-17T02:22:05.135564+0000","last_undegraded":"2024-08-17T02:22:05.135564+0000","last_fullsized":"2024-08-17T02:22:05.135564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:00:22.156028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.115878+0000","last_change":"2024-08-17T02:22:04.129312+0000","last_active":"2024-08-17T02:22:05.115878+0000","last_peered":"2024-08-17T02:22:05.115878+0000","last_clean":"2024-08-17T02:22:05.115878+0000","last_became_active":"2024-08-17T02:22:04.129003+0000","last_became_peered":"2024-08-17T02:22:04.129003+0000","last_unstale":"2024-08-17T02:22:05.115878+0000","last_undegraded":"2024-08-17T02:22:05.115878+0000","last_fullsized":"2024-08-17T02:22:05.115878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139117+0000","last_change":"2024-08-17T02:22:05.133370+0000","last_active":"2024-08-17T02:22:05.139117+0000","last_peered":"2024-08-17T02:22:05.139117+0000","last_clean":"2024-08-17T02:22:05.139117+0000","last_became_active":"2024-08-17T02:22:05.133041+0000","last_became_peered":"2024-08-17T02:22:05.133041+0000","last_unstale":"2024-08-17T02:22:05.139117+0000","last_undegraded":"2024-08-17T02:22:05.139117+0000","last_fullsized":"2024-08-17T02:22:05.139117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120010+0000","last_change":"2024-08-17T02:22:04.134772+0000","last_active":"2024-08-17T02:22:05.120010+0000","last_peered":"2024-08-17T02:22:05.120010+0000","last_clean":"2024-08-17T02:22:05.120010+0000","last_became_active":"2024-08-17T02:22:04.134509+0000","last_became_peered":"2024-08-17T02:22:04.134509+0000","last_unstale":"2024-08-17T02:22:05.120010+0000","last_undegraded":"2024-08-17T02:22:05.120010+0000","last_fullsized":"2024-08-17T02:22:05.120010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:01.702706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142478+0000","last_change":"2024-08-17T02:22:05.139006+0000","last_active":"2024-08-17T02:22:05.142478+0000","last_peered":"2024-08-17T02:22:05.142478+0000","last_clean":"2024-08-17T02:22:05.142478+0000","last_became_active":"2024-08-17T02:22:05.138609+0000","last_became_peered":"2024-08-17T02:22:05.138609+0000","last_unstale":"2024-08-17T02:22:05.142478+0000","last_undegraded":"2024-08-17T02:22:05.142478+0000","last_fullsized":"2024-08-17T02:22:05.142478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.116705+0000","last_change":"2024-08-17T02:22:04.129251+0000","last_active":"2024-08-17T02:22:05.116705+0000","last_peered":"2024-08-17T02:22:05.116705+0000","last_clean":"2024-08-17T02:22:05.116705+0000","last_became_active":"2024-08-17T02:22:04.126603+0000","last_became_peered":"2024-08-17T02:22:04.126603+0000","last_unstale":"2024-08-17T02:22:05.116705+0000","last_undegraded":"2024-08-17T02:22:05.116705+0000","last_fullsized":"2024-08-17T02:22:05.116705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139326+0000","last_change":"2024-08-17T02:22:05.131334+0000","last_active":"2024-08-17T02:22:05.139326+0000","last_peered":"2024-08-17T02:22:05.139326+0000","last_clean":"2024-08-17T02:22:05.139326+0000","last_became_active":"2024-08-17T02:22:05.130325+0000","last_became_peered":"2024-08-17T02:22:05.130325+0000","last_unstale":"2024-08-17T02:22:05.139326+0000","last_undegraded":"2024-08-17T02:22:05.139326+0000","last_fullsized":"2024-08-17T02:22:05.139326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:47:26.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142458+0000","last_change":"2024-08-17T02:22:04.135036+0000","last_active":"2024-08-17T02:22:05.142458+0000","last_peered":"2024-08-17T02:22:05.142458+0000","last_clean":"2024-08-17T02:22:05.142458+0000","last_became_active":"2024-08-17T02:22:04.134786+0000","last_became_peered":"2024-08-17T02:22:04.134786+0000","last_unstale":"2024-08-17T02:22:05.142458+0000","last_undegraded":"2024-08-17T02:22:05.142458+0000","last_fullsized":"2024-08-17T02:22:05.142458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.997912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146826+0000","last_change":"2024-08-17T02:22:05.141338+0000","last_active":"2024-08-17T02:22:05.146826+0000","last_peered":"2024-08-17T02:22:05.146826+0000","last_clean":"2024-08-17T02:22:05.146826+0000","last_became_active":"2024-08-17T02:22:05.139395+0000","last_became_peered":"2024-08-17T02:22:05.139395+0000","last_unstale":"2024-08-17T02:22:05.146826+0000","last_undegraded":"2024-08-17T02:22:05.146826+0000","last_fullsized":"2024-08-17T02:22:05.146826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:07.057717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146729+0000","last_change":"2024-08-17T02:22:04.127992+0000","last_active":"2024-08-17T02:22:05.146729+0000","last_peered":"2024-08-17T02:22:05.146729+0000","last_clean":"2024-08-17T02:22:05.146729+0000","last_became_active":"2024-08-17T02:22:04.127656+0000","last_became_peered":"2024-08-17T02:22:04.127656+0000","last_unstale":"2024-08-17T02:22:05.146729+0000","last_undegraded":"2024-08-17T02:22:05.146729+0000","last_fullsized":"2024-08-17T02:22:05.146729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:06.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142876+0000","last_change":"2024-08-17T02:22:05.136992+0000","last_active":"2024-08-17T02:22:05.142876+0000","last_peered":"2024-08-17T02:22:05.142876+0000","last_clean":"2024-08-17T02:22:05.142876+0000","last_became_active":"2024-08-17T02:22:05.136611+0000","last_became_peered":"2024-08-17T02:22:05.136611+0000","last_unstale":"2024-08-17T02:22:05.142876+0000","last_undegraded":"2024-08-17T02:22:05.142876+0000","last_fullsized":"2024-08-17T02:22:05.142876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:27:57.075296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147451+0000","last_change":"2024-08-17T02:22:04.136005+0000","last_active":"2024-08-17T02:22:05.147451+0000","last_peered":"2024-08-17T02:22:05.147451+0000","last_clean":"2024-08-17T02:22:05.147451+0000","last_became_active":"2024-08-17T02:22:04.135718+0000","last_became_peered":"2024-08-17T02:22:04.135718+0000","last_unstale":"2024-08-17T02:22:05.147451+0000","last_undegraded":"2024-08-17T02:22:05.147451+0000","last_fullsized":"2024-08-17T02:22:05.147451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:11.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144007+0000","last_change":"2024-08-17T02:22:05.137445+0000","last_active":"2024-08-17T02:22:05.144007+0000","last_peered":"2024-08-17T02:22:05.144007+0000","last_clean":"2024-08-17T02:22:05.144007+0000","last_became_active":"2024-08-17T02:22:05.137047+0000","last_became_peered":"2024-08-17T02:22:05.137047+0000","last_unstale":"2024-08-17T02:22:05.144007+0000","last_undegraded":"2024-08-17T02:22:05.144007+0000","last_fullsized":"2024-08-17T02:22:05.144007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:49.930124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.117051+0000","last_change":"2024-08-17T02:22:04.128882+0000","last_active":"2024-08-17T02:22:05.117051+0000","last_peered":"2024-08-17T02:22:05.117051+0000","last_clean":"2024-08-17T02:22:05.117051+0000","last_became_active":"2024-08-17T02:22:04.124521+0000","last_became_peered":"2024-08-17T02:22:04.124521+0000","last_unstale":"2024-08-17T02:22:05.117051+0000","last_undegraded":"2024-08-17T02:22:05.117051+0000","last_fullsized":"2024-08-17T02:22:05.117051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146565+0000","last_change":"2024-08-17T02:22:05.140259+0000","last_active":"2024-08-17T02:22:05.146565+0000","last_peered":"2024-08-17T02:22:05.146565+0000","last_clean":"2024-08-17T02:22:05.146565+0000","last_became_active":"2024-08-17T02:22:05.139966+0000","last_became_peered":"2024-08-17T02:22:05.139966+0000","last_unstale":"2024-08-17T02:22:05.146565+0000","last_undegraded":"2024-08-17T02:22:05.146565+0000","last_fullsized":"2024-08-17T02:22:05.146565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:01:27.557725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120796+0000","last_change":"2024-08-17T02:22:04.122812+0000","last_active":"2024-08-17T02:22:05.120796+0000","last_peered":"2024-08-17T02:22:05.120796+0000","last_clean":"2024-08-17T02:22:05.120796+0000","last_became_active":"2024-08-17T02:22:04.122506+0000","last_became_peered":"2024-08-17T02:22:04.122506+0000","last_unstale":"2024-08-17T02:22:05.120796+0000","last_undegraded":"2024-08-17T02:22:05.120796+0000","last_fullsized":"2024-08-17T02:22:05.120796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:56.324601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133838+0000","last_change":"2024-08-17T02:22:05.131385+0000","last_active":"2024-08-17T02:22:05.133838+0000","last_peered":"2024-08-17T02:22:05.133838+0000","last_clean":"2024-08-17T02:22:05.133838+0000","last_became_active":"2024-08-17T02:22:05.130986+0000","last_became_peered":"2024-08-17T02:22:05.130986+0000","last_unstale":"2024-08-17T02:22:05.133838+0000","last_undegraded":"2024-08-17T02:22:05.133838+0000","last_fullsized":"2024-08-17T02:22:05.133838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.132407+0000","last_change":"2024-08-17T02:22:04.125802+0000","last_active":"2024-08-17T02:22:05.132407+0000","last_peered":"2024-08-17T02:22:05.132407+0000","last_clean":"2024-08-17T02:22:05.132407+0000","last_became_active":"2024-08-17T02:22:04.125462+0000","last_became_peered":"2024-08-17T02:22:04.125462+0000","last_unstale":"2024-08-17T02:22:05.132407+0000","last_undegraded":"2024-08-17T02:22:05.132407+0000","last_fullsized":"2024-08-17T02:22:05.132407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143008+0000","last_change":"2024-08-17T02:22:05.137082+0000","last_active":"2024-08-17T02:22:05.143008+0000","last_peered":"2024-08-17T02:22:05.143008+0000","last_clean":"2024-08-17T02:22:05.143008+0000","last_became_active":"2024-08-17T02:22:05.136773+0000","last_became_peered":"2024-08-17T02:22:05.136773+0000","last_unstale":"2024-08-17T02:22:05.143008+0000","last_undegraded":"2024-08-17T02:22:05.143008+0000","last_fullsized":"2024-08-17T02:22:05.143008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142302+0000","last_change":"2024-08-17T02:22:04.136447+0000","last_active":"2024-08-17T02:22:05.142302+0000","last_peered":"2024-08-17T02:22:05.142302+0000","last_clean":"2024-08-17T02:22:05.142302+0000","last_became_active":"2024-08-17T02:22:04.136105+0000","last_became_peered":"2024-08-17T02:22:04.136105+0000","last_unstale":"2024-08-17T02:22:05.142302+0000","last_undegraded":"2024-08-17T02:22:05.142302+0000","last_fullsized":"2024-08-17T02:22:05.142302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147858+0000","last_change":"2024-08-17T02:22:05.145138+0000","last_active":"2024-08-17T02:22:05.147858+0000","last_peered":"2024-08-17T02:22:05.147858+0000","last_clean":"2024-08-17T02:22:05.147858+0000","last_became_active":"2024-08-17T02:22:05.144754+0000","last_became_peered":"2024-08-17T02:22:05.144754+0000","last_unstale":"2024-08-17T02:22:05.147858+0000","last_undegraded":"2024-08-17T02:22:05.147858+0000","last_fullsized":"2024-08-17T02:22:05.147858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:36:55.030030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133637+0000","last_change":"2024-08-17T02:22:04.128289+0000","last_active":"2024-08-17T02:22:05.133637+0000","last_peered":"2024-08-17T02:22:05.133637+0000","last_clean":"2024-08-17T02:22:05.133637+0000","last_became_active":"2024-08-17T02:22:04.127984+0000","last_became_peered":"2024-08-17T02:22:04.127984+0000","last_unstale":"2024-08-17T02:22:05.133637+0000","last_undegraded":"2024-08-17T02:22:05.133637+0000","last_fullsized":"2024-08-17T02:22:05.133637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147852+0000","last_change":"2024-08-17T02:22:05.145108+0000","last_active":"2024-08-17T02:22:05.147852+0000","last_peered":"2024-08-17T02:22:05.147852+0000","last_clean":"2024-08-17T02:22:05.147852+0000","last_became_active":"2024-08-17T02:22:05.144771+0000","last_became_peered":"2024-08-17T02:22:05.144771+0000","last_unstale":"2024-08-17T02:22:05.147852+0000","last_undegraded":"2024-08-17T02:22:05.147852+0000","last_fullsized":"2024-08-17T02:22:05.147852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:34:43.102929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139054+0000","last_change":"2024-08-17T02:22:04.125374+0000","last_active":"2024-08-17T02:22:05.139054+0000","last_peered":"2024-08-17T02:22:05.139054+0000","last_clean":"2024-08-17T02:22:05.139054+0000","last_became_active":"2024-08-17T02:22:04.125059+0000","last_became_peered":"2024-08-17T02:22:04.125059+0000","last_unstale":"2024-08-17T02:22:05.139054+0000","last_undegraded":"2024-08-17T02:22:05.139054+0000","last_fullsized":"2024-08-17T02:22:05.139054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:01.702706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142758+0000","last_change":"2024-08-17T02:22:05.139051+0000","last_active":"2024-08-17T02:22:05.142758+0000","last_peered":"2024-08-17T02:22:05.142758+0000","last_clean":"2024-08-17T02:22:05.142758+0000","last_became_active":"2024-08-17T02:22:05.135262+0000","last_became_peered":"2024-08-17T02:22:05.135262+0000","last_unstale":"2024-08-17T02:22:05.142758+0000","last_undegraded":"2024-08-17T02:22:05.142758+0000","last_fullsized":"2024-08-17T02:22:05.142758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:42:24.349708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.118431+0000","last_change":"2024-08-17T02:22:04.137875+0000","last_active":"2024-08-17T02:22:05.118431+0000","last_peered":"2024-08-17T02:22:05.118431+0000","last_clean":"2024-08-17T02:22:05.118431+0000","last_became_active":"2024-08-17T02:22:04.137590+0000","last_became_peered":"2024-08-17T02:22:04.137590+0000","last_unstale":"2024-08-17T02:22:05.118431+0000","last_undegraded":"2024-08-17T02:22:05.118431+0000","last_fullsized":"2024-08-17T02:22:05.118431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146797+0000","last_change":"2024-08-17T02:22:05.142646+0000","last_active":"2024-08-17T02:22:05.146797+0000","last_peered":"2024-08-17T02:22:05.146797+0000","last_clean":"2024-08-17T02:22:05.146797+0000","last_became_active":"2024-08-17T02:22:05.141915+0000","last_became_peered":"2024-08-17T02:22:05.141915+0000","last_unstale":"2024-08-17T02:22:05.146797+0000","last_undegraded":"2024-08-17T02:22:05.146797+0000","last_fullsized":"2024-08-17T02:22:05.146797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:30:37.267266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120860+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:05.120860+0000","last_peered":"2024-08-17T02:22:05.120860+0000","last_clean":"2024-08-17T02:22:05.120860+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:05.120860+0000","last_undegraded":"2024-08-17T02:22:05.120860+0000","last_fullsized":"2024-08-17T02:22:05.120860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.116535+0000","last_change":"2024-08-17T02:22:04.129059+0000","last_active":"2024-08-17T02:22:05.116535+0000","last_peered":"2024-08-17T02:22:05.116535+0000","last_clean":"2024-08-17T02:22:05.116535+0000","last_became_active":"2024-08-17T02:22:04.126173+0000","last_became_peered":"2024-08-17T02:22:04.126173+0000","last_unstale":"2024-08-17T02:22:05.116535+0000","last_undegraded":"2024-08-17T02:22:05.116535+0000","last_fullsized":"2024-08-17T02:22:05.116535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.145732+0000","last_change":"2024-08-17T02:22:05.142911+0000","last_active":"2024-08-17T02:22:05.145732+0000","last_peered":"2024-08-17T02:22:05.145732+0000","last_clean":"2024-08-17T02:22:05.145732+0000","last_became_active":"2024-08-17T02:22:05.142665+0000","last_became_peered":"2024-08-17T02:22:05.142665+0000","last_unstale":"2024-08-17T02:22:05.145732+0000","last_undegraded":"2024-08-17T02:22:05.145732+0000","last_fullsized":"2024-08-17T02:22:05.145732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:41:42.553817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120935+0000","last_change":"2024-08-17T02:22:04.125465+0000","last_active":"2024-08-17T02:22:05.120935+0000","last_peered":"2024-08-17T02:22:05.120935+0000","last_clean":"2024-08-17T02:22:05.120935+0000","last_became_active":"2024-08-17T02:22:04.125205+0000","last_became_peered":"2024-08-17T02:22:04.125205+0000","last_unstale":"2024-08-17T02:22:05.120935+0000","last_undegraded":"2024-08-17T02:22:05.120935+0000","last_fullsized":"2024-08-17T02:22:05.120935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:59:53.518892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139508+0000","last_change":"2024-08-17T02:22:05.132007+0000","last_active":"2024-08-17T02:22:05.139508+0000","last_peered":"2024-08-17T02:22:05.139508+0000","last_clean":"2024-08-17T02:22:05.139508+0000","last_became_active":"2024-08-17T02:22:05.131083+0000","last_became_peered":"2024-08-17T02:22:05.131083+0000","last_unstale":"2024-08-17T02:22:05.139508+0000","last_undegraded":"2024-08-17T02:22:05.139508+0000","last_fullsized":"2024-08-17T02:22:05.139508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:15.303455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.119774+0000","last_change":"2024-08-17T02:22:04.128675+0000","last_active":"2024-08-17T02:22:05.119774+0000","last_peered":"2024-08-17T02:22:05.119774+0000","last_clean":"2024-08-17T02:22:05.119774+0000","last_became_active":"2024-08-17T02:22:04.128264+0000","last_became_peered":"2024-08-17T02:22:04.128264+0000","last_unstale":"2024-08-17T02:22:05.119774+0000","last_undegraded":"2024-08-17T02:22:05.119774+0000","last_fullsized":"2024-08-17T02:22:05.119774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.125552+0000","last_change":"2024-08-17T02:22:05.123180+0000","last_active":"2024-08-17T02:22:05.125552+0000","last_peered":"2024-08-17T02:22:05.125552+0000","last_clean":"2024-08-17T02:22:05.125552+0000","last_became_active":"2024-08-17T02:22:05.122887+0000","last_became_peered":"2024-08-17T02:22:05.122887+0000","last_unstale":"2024-08-17T02:22:05.125552+0000","last_undegraded":"2024-08-17T02:22:05.125552+0000","last_fullsized":"2024-08-17T02:22:05.125552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.117161+0000","last_change":"2024-08-17T02:22:04.125809+0000","last_active":"2024-08-17T02:22:05.117161+0000","last_peered":"2024-08-17T02:22:05.117161+0000","last_clean":"2024-08-17T02:22:05.117161+0000","last_became_active":"2024-08-17T02:22:04.125484+0000","last_became_peered":"2024-08-17T02:22:04.125484+0000","last_unstale":"2024-08-17T02:22:05.117161+0000","last_undegraded":"2024-08-17T02:22:05.117161+0000","last_fullsized":"2024-08-17T02:22:05.117161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146600+0000","last_change":"2024-08-17T02:22:05.138522+0000","last_active":"2024-08-17T02:22:05.146600+0000","last_peered":"2024-08-17T02:22:05.146600+0000","last_clean":"2024-08-17T02:22:05.146600+0000","last_became_active":"2024-08-17T02:22:05.136133+0000","last_became_peered":"2024-08-17T02:22:05.136133+0000","last_unstale":"2024-08-17T02:22:05.146600+0000","last_undegraded":"2024-08-17T02:22:05.146600+0000","last_fullsized":"2024-08-17T02:22:05.146600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:48.803891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110226+0000","last_change":"2024-08-17T02:22:04.139328+0000","last_active":"2024-08-17T02:22:05.110226+0000","last_peered":"2024-08-17T02:22:05.110226+0000","last_clean":"2024-08-17T02:22:05.110226+0000","last_became_active":"2024-08-17T02:22:04.139078+0000","last_became_peered":"2024-08-17T02:22:04.139078+0000","last_unstale":"2024-08-17T02:22:05.110226+0000","last_undegraded":"2024-08-17T02:22:05.110226+0000","last_fullsized":"2024-08-17T02:22:05.110226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:26:59.050552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141791+0000","last_change":"2024-08-17T02:22:05.135288+0000","last_active":"2024-08-17T02:22:05.141791+0000","last_peered":"2024-08-17T02:22:05.141791+0000","last_clean":"2024-08-17T02:22:05.141791+0000","last_became_active":"2024-08-17T02:22:05.134902+0000","last_became_peered":"2024-08-17T02:22:05.134902+0000","last_unstale":"2024-08-17T02:22:05.141791+0000","last_undegraded":"2024-08-17T02:22:05.141791+0000","last_fullsized":"2024-08-17T02:22:05.141791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:41:22.363365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.127565+0000","last_change":"2024-08-17T02:22:04.136037+0000","last_active":"2024-08-17T02:22:05.127565+0000","last_peered":"2024-08-17T02:22:05.127565+0000","last_clean":"2024-08-17T02:22:05.127565+0000","last_became_active":"2024-08-17T02:22:04.135795+0000","last_became_peered":"2024-08-17T02:22:04.135795+0000","last_unstale":"2024-08-17T02:22:05.127565+0000","last_undegraded":"2024-08-17T02:22:05.127565+0000","last_fullsized":"2024-08-17T02:22:05.127565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.124422+0000","last_change":"2024-08-17T02:22:05.123134+0000","last_active":"2024-08-17T02:22:05.124422+0000","last_peered":"2024-08-17T02:22:05.124422+0000","last_clean":"2024-08-17T02:22:05.124422+0000","last_became_active":"2024-08-17T02:22:05.122849+0000","last_became_peered":"2024-08-17T02:22:05.122849+0000","last_unstale":"2024-08-17T02:22:05.124422+0000","last_undegraded":"2024-08-17T02:22:05.124422+0000","last_fullsized":"2024-08-17T02:22:05.124422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:37.064590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-08-17T02:22:07.113057+0000","last_change":"2024-08-17T02:22:04.124830+0000","last_active":"2024-08-17T02:22:07.113057+0000","last_peered":"2024-08-17T02:22:07.113057+0000","last_clean":"2024-08-17T02:22:07.113057+0000","last_became_active":"2024-08-17T02:22:04.124412+0000","last_became_peered":"2024-08-17T02:22:04.124412+0000","last_unstale":"2024-08-17T02:22:07.113057+0000","last_undegraded":"2024-08-17T02:22:07.113057+0000","last_fullsized":"2024-08-17T02:22:07.113057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:06.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.145579+0000","last_change":"2024-08-17T02:22:05.140560+0000","last_active":"2024-08-17T02:22:05.145579+0000","last_peered":"2024-08-17T02:22:05.145579+0000","last_clean":"2024-08-17T02:22:05.145579+0000","last_became_active":"2024-08-17T02:22:05.140249+0000","last_became_peered":"2024-08-17T02:22:05.140249+0000","last_unstale":"2024-08-17T02:22:05.145579+0000","last_undegraded":"2024-08-17T02:22:05.145579+0000","last_fullsized":"2024-08-17T02:22:05.145579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:24:46.183039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138031+0000","last_change":"2024-08-17T02:22:04.140288+0000","last_active":"2024-08-17T02:22:05.138031+0000","last_peered":"2024-08-17T02:22:05.138031+0000","last_clean":"2024-08-17T02:22:05.138031+0000","last_became_active":"2024-08-17T02:22:04.139999+0000","last_became_peered":"2024-08-17T02:22:04.139999+0000","last_unstale":"2024-08-17T02:22:05.138031+0000","last_undegraded":"2024-08-17T02:22:05.138031+0000","last_fullsized":"2024-08-17T02:22:05.138031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.997912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136517+0000","last_change":"2024-08-17T02:22:05.129259+0000","last_active":"2024-08-17T02:22:05.136517+0000","last_peered":"2024-08-17T02:22:05.136517+0000","last_clean":"2024-08-17T02:22:05.136517+0000","last_became_active":"2024-08-17T02:22:05.128993+0000","last_became_peered":"2024-08-17T02:22:05.128993+0000","last_unstale":"2024-08-17T02:22:05.136517+0000","last_undegraded":"2024-08-17T02:22:05.136517+0000","last_fullsized":"2024-08-17T02:22:05.136517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120637+0000","last_change":"2024-08-17T02:22:04.136465+0000","last_active":"2024-08-17T02:22:05.120637+0000","last_peered":"2024-08-17T02:22:05.120637+0000","last_clean":"2024-08-17T02:22:05.120637+0000","last_became_active":"2024-08-17T02:22:04.136153+0000","last_became_peered":"2024-08-17T02:22:04.136153+0000","last_unstale":"2024-08-17T02:22:05.120637+0000","last_undegraded":"2024-08-17T02:22:05.120637+0000","last_fullsized":"2024-08-17T02:22:05.120637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:25:44.685521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.130033+0000","last_change":"2024-08-17T02:22:05.125665+0000","last_active":"2024-08-17T02:22:05.130033+0000","last_peered":"2024-08-17T02:22:05.130033+0000","last_clean":"2024-08-17T02:22:05.130033+0000","last_became_active":"2024-08-17T02:22:05.125115+0000","last_became_peered":"2024-08-17T02:22:05.125115+0000","last_unstale":"2024-08-17T02:22:05.130033+0000","last_undegraded":"2024-08-17T02:22:05.130033+0000","last_fullsized":"2024-08-17T02:22:05.130033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:57.328385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110749+0000","last_change":"2024-08-17T02:22:04.125704+0000","last_active":"2024-08-17T02:22:05.110749+0000","last_peered":"2024-08-17T02:22:05.110749+0000","last_clean":"2024-08-17T02:22:05.110749+0000","last_became_active":"2024-08-17T02:22:04.124384+0000","last_became_peered":"2024-08-17T02:22:04.124384+0000","last_unstale":"2024-08-17T02:22:05.110749+0000","last_undegraded":"2024-08-17T02:22:05.110749+0000","last_fullsized":"2024-08-17T02:22:05.110749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:56.324601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144755+0000","last_change":"2024-08-17T02:22:05.140080+0000","last_active":"2024-08-17T02:22:05.144755+0000","last_peered":"2024-08-17T02:22:05.144755+0000","last_clean":"2024-08-17T02:22:05.144755+0000","last_became_active":"2024-08-17T02:22:05.139642+0000","last_became_peered":"2024-08-17T02:22:05.139642+0000","last_unstale":"2024-08-17T02:22:05.144755+0000","last_undegraded":"2024-08-17T02:22:05.144755+0000","last_fullsized":"2024-08-17T02:22:05.144755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110708+0000","last_change":"2024-08-17T02:22:04.128418+0000","last_active":"2024-08-17T02:22:05.110708+0000","last_peered":"2024-08-17T02:22:05.110708+0000","last_clean":"2024-08-17T02:22:05.110708+0000","last_became_active":"2024-08-17T02:22:04.128178+0000","last_became_peered":"2024-08-17T02:22:04.128178+0000","last_unstale":"2024-08-17T02:22:05.110708+0000","last_undegraded":"2024-08-17T02:22:05.110708+0000","last_fullsized":"2024-08-17T02:22:05.110708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135568+0000","last_change":"2024-08-17T02:22:05.132678+0000","last_active":"2024-08-17T02:22:05.135568+0000","last_peered":"2024-08-17T02:22:05.135568+0000","last_clean":"2024-08-17T02:22:05.135568+0000","last_became_active":"2024-08-17T02:22:05.132375+0000","last_became_peered":"2024-08-17T02:22:05.132375+0000","last_unstale":"2024-08-17T02:22:05.135568+0000","last_undegraded":"2024-08-17T02:22:05.135568+0000","last_fullsized":"2024-08-17T02:22:05.135568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:05:58.104311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144474+0000","last_change":"2024-08-17T02:22:04.120559+0000","last_active":"2024-08-17T02:22:05.144474+0000","last_peered":"2024-08-17T02:22:05.144474+0000","last_clean":"2024-08-17T02:22:05.144474+0000","last_became_active":"2024-08-17T02:22:04.120229+0000","last_became_peered":"2024-08-17T02:22:04.120229+0000","last_unstale":"2024-08-17T02:22:05.144474+0000","last_undegraded":"2024-08-17T02:22:05.144474+0000","last_fullsized":"2024-08-17T02:22:05.144474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.464171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133762+0000","last_change":"2024-08-17T02:22:05.129982+0000","last_active":"2024-08-17T02:22:05.133762+0000","last_peered":"2024-08-17T02:22:05.133762+0000","last_clean":"2024-08-17T02:22:05.133762+0000","last_became_active":"2024-08-17T02:22:05.129645+0000","last_became_peered":"2024-08-17T02:22:05.129645+0000","last_unstale":"2024-08-17T02:22:05.133762+0000","last_undegraded":"2024-08-17T02:22:05.133762+0000","last_fullsized":"2024-08-17T02:22:05.133762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141475+0000","last_change":"2024-08-17T02:22:05.130378+0000","last_active":"2024-08-17T02:22:05.141475+0000","last_peered":"2024-08-17T02:22:05.141475+0000","last_clean":"2024-08-17T02:22:05.141475+0000","last_became_active":"2024-08-17T02:22:05.130027+0000","last_became_peered":"2024-08-17T02:22:05.130027+0000","last_unstale":"2024-08-17T02:22:05.141475+0000","last_undegraded":"2024-08-17T02:22:05.141475+0000","last_fullsized":"2024-08-17T02:22:05.141475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:44:40.942386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136607+0000","last_change":"2024-08-17T02:22:04.126856+0000","last_active":"2024-08-17T02:22:05.136607+0000","last_peered":"2024-08-17T02:22:05.136607+0000","last_clean":"2024-08-17T02:22:05.136607+0000","last_became_active":"2024-08-17T02:22:04.126479+0000","last_became_peered":"2024-08-17T02:22:04.126479+0000","last_unstale":"2024-08-17T02:22:05.136607+0000","last_undegraded":"2024-08-17T02:22:05.136607+0000","last_fullsized":"2024-08-17T02:22:05.136607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139018+0000","last_change":"2024-08-17T02:22:05.129249+0000","last_active":"2024-08-17T02:22:05.139018+0000","last_peered":"2024-08-17T02:22:05.139018+0000","last_clean":"2024-08-17T02:22:05.139018+0000","last_became_active":"2024-08-17T02:22:05.127468+0000","last_became_peered":"2024-08-17T02:22:05.127468+0000","last_unstale":"2024-08-17T02:22:05.139018+0000","last_undegraded":"2024-08-17T02:22:05.139018+0000","last_fullsized":"2024-08-17T02:22:05.139018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:48:12.922294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.137386+0000","last_change":"2024-08-17T02:22:04.130183+0000","last_active":"2024-08-17T02:22:05.137386+0000","last_peered":"2024-08-17T02:22:05.137386+0000","last_clean":"2024-08-17T02:22:05.137386+0000","last_became_active":"2024-08-17T02:22:04.129922+0000","last_became_peered":"2024-08-17T02:22:04.129922+0000","last_unstale":"2024-08-17T02:22:05.137386+0000","last_undegraded":"2024-08-17T02:22:05.137386+0000","last_fullsized":"2024-08-17T02:22:05.137386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139349+0000","last_change":"2024-08-17T02:22:05.131559+0000","last_active":"2024-08-17T02:22:05.139349+0000","last_peered":"2024-08-17T02:22:05.139349+0000","last_clean":"2024-08-17T02:22:05.139349+0000","last_became_active":"2024-08-17T02:22:05.131142+0000","last_became_peered":"2024-08-17T02:22:05.131142+0000","last_unstale":"2024-08-17T02:22:05.139349+0000","last_undegraded":"2024-08-17T02:22:05.139349+0000","last_fullsized":"2024-08-17T02:22:05.139349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:21:52.626562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134704+0000","last_change":"2024-08-17T02:22:04.136408+0000","last_active":"2024-08-17T02:22:05.134704+0000","last_peered":"2024-08-17T02:22:05.134704+0000","last_clean":"2024-08-17T02:22:05.134704+0000","last_became_active":"2024-08-17T02:22:04.136140+0000","last_became_peered":"2024-08-17T02:22:04.136140+0000","last_unstale":"2024-08-17T02:22:05.134704+0000","last_undegraded":"2024-08-17T02:22:05.134704+0000","last_fullsized":"2024-08-17T02:22:05.134704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138499+0000","last_change":"2024-08-17T02:22:05.134585+0000","last_active":"2024-08-17T02:22:05.138499+0000","last_peered":"2024-08-17T02:22:05.138499+0000","last_clean":"2024-08-17T02:22:05.138499+0000","last_became_active":"2024-08-17T02:22:05.134230+0000","last_became_peered":"2024-08-17T02:22:05.134230+0000","last_unstale":"2024-08-17T02:22:05.138499+0000","last_undegraded":"2024-08-17T02:22:05.138499+0000","last_fullsized":"2024-08-17T02:22:05.138499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144051+0000","last_change":"2024-08-17T02:22:04.140683+0000","last_active":"2024-08-17T02:22:05.144051+0000","last_peered":"2024-08-17T02:22:05.144051+0000","last_clean":"2024-08-17T02:22:05.144051+0000","last_became_active":"2024-08-17T02:22:04.140386+0000","last_became_peered":"2024-08-17T02:22:04.140386+0000","last_unstale":"2024-08-17T02:22:05.144051+0000","last_undegraded":"2024-08-17T02:22:05.144051+0000","last_fullsized":"2024-08-17T02:22:05.144051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298360,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073480,"statfs":{"total":96636764160,"available":96331243520,"internally_reserved":0,"allocated":823296,"data_stored":669312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":237568,"data_stored":79926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":229376,"data_stored":78464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":237568,"data_stored":78500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.415}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.544}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":237568,"data_stored":79926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":78374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":233472,"data_stored":78534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":831488,"data_stored":669368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:16.518 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:16.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.608+0000 7ff793fff700 1 Processor -- start 2024-08-17T02:22:16.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.608+0000 7ff793fff700 1 -- start start 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff794159040 0x7ff7940cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79415d5b0 0x7ff7940d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7940cf190 con 0x7ff7940d0010 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7940d09a0 con 0x7ff794159040 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940ce5e0 con 0x7ff79415d5b0 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff7927fc700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff7940d0010 0x7ff79415cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50924/0 (socket says 172.21.15.100:50924) 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff7927fc700 1 -- 172.21.15.100:0/1038734345 learned_addr learned my addr 172.21.15.100:0/1038734345 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff792ffd700 1 --2- 172.21.15.100:0/1038734345 >> v2:172.21.15.100:3300/0 conn(0x7ff794159040 0x7ff7940cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff7937fe700 1 --2- 172.21.15.100:0/1038734345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79415d5b0 0x7ff7940d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff791ffb700 1 -- 172.21.15.100:0/1038734345 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643695281 0 0) 0x7ff7940cf190 con 0x7ff7940d0010 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff791ffb700 1 -- 172.21.15.100:0/1038734345 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff77c003570 con 0x7ff7940d0010 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff792ffd700 1 -- 172.21.15.100:0/1038734345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79415d5b0 msgr2=0x7ff7940d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff792ffd700 1 --2- 172.21.15.100:0/1038734345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79415d5b0 0x7ff7940d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff792ffd700 1 -- 172.21.15.100:0/1038734345 >> v1:172.21.15.100:6789/0 conn(0x7ff7940d0010 legacy=0x7ff79415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff792ffd700 1 -- 172.21.15.100:0/1038734345 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7940ce860 con 0x7ff794159040 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff7937fe700 1 --2- 172.21.15.100:0/1038734345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79415d5b0 0x7ff7940d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff792ffd700 1 --2- 172.21.15.100:0/1038734345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794159040 0x7ff7940cf620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7840098e0 tx=0x7ff7840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badad1bf7c986cb6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff791ffb700 1 -- 172.21.15.100:0/1038734345 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784011630 con 0x7ff794159040 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff791ffb700 1 -- 172.21.15.100:0/1038734345 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff784011790 con 0x7ff794159040 2024-08-17T02:22:16.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff791ffb700 1 -- 172.21.15.100:0/1038734345 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784011b10 con 0x7ff794159040 2024-08-17T02:22:16.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 -- 172.21.15.100:0/1038734345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794159040 msgr2=0x7ff7940cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 --2- 172.21.15.100:0/1038734345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794159040 0x7ff7940cf620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7840098e0 tx=0x7ff7840091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 -- 172.21.15.100:0/1038734345 shutdown_connections 2024-08-17T02:22:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 --2- 172.21.15.100:0/1038734345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79415d5b0 0x7ff7940d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 --2- 172.21.15.100:0/1038734345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794159040 0x7ff7940cf620 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 -- 172.21.15.100:0/1038734345 >> 172.21.15.100:0/1038734345 conn(0x7ff794068250 msgr2=0x7ff794068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 -- 172.21.15.100:0/1038734345 shutdown_connections 2024-08-17T02:22:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.612+0000 7ff793fff700 1 -- 172.21.15.100:0/1038734345 wait complete. 2024-08-17T02:22:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 Processor -- start 2024-08-17T02:22:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 -- start start 2024-08-17T02:22:16.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7940d0010 0x7ff7940b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff794159040 0x7ff7940b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff79415d5b0 0x7ff794150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940b0b30 con 0x7ff79415d5b0 2024-08-17T02:22:16.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940aff20 con 0x7ff794159040 2024-08-17T02:22:16.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940b0200 con 0x7ff7940d0010 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff79415d5b0 0x7ff794150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff79415d5b0 0x7ff794150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52524/0 (socket says 172.21.15.100:52524) 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 -- 172.21.15.100:0/1397464668 learned_addr learned my addr 172.21.15.100:0/1397464668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff792ffd700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7940d0010 0x7ff7940b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7927fc700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff794159040 0x7ff7940b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 -- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7940d0010 msgr2=0x7ff7940b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7940d0010 0x7ff7940b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 -- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff794159040 msgr2=0x7ff7940b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff794159040 0x7ff7940b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 -- 172.21.15.100:0/1397464668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c009a20 con 0x7ff79415d5b0 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7927fc700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff794159040 0x7ff7940b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff7937fe700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff79415d5b0 0x7ff794150730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009fc0 tx=0x7ff78c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff77affd700 1 -- 172.21.15.100:0/1397464668 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00cd30 con 0x7ff79415d5b0 2024-08-17T02:22:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff77affd700 1 -- 172.21.15.100:0/1397464668 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c00f040 con 0x7ff79415d5b0 2024-08-17T02:22:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff784009420 con 0x7ff79415d5b0 2024-08-17T02:22:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff77affd700 1 -- 172.21.15.100:0/1397464668 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c0146d0 con 0x7ff79415d5b0 2024-08-17T02:22:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.616+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7940b07c0 con 0x7ff79415d5b0 2024-08-17T02:22:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.620+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7940b0ca0 con 0x7ff79415d5b0 2024-08-17T02:22:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.620+0000 7ff77affd700 1 -- 172.21.15.100:0/1397464668 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff78c003020 con 0x7ff79415d5b0 2024-08-17T02:22:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.620+0000 7ff77affd700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff780060860 0x7ff780062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.620+0000 7ff77affd700 1 -- 172.21.15.100:0/1397464668 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff78c07d600 con 0x7ff79415d5b0 2024-08-17T02:22:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.620+0000 7ff792ffd700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff780060860 0x7ff780062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.620+0000 7ff792ffd700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff780060860 0x7ff780062d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7840091e0 tx=0x7ff784017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.628+0000 7ff77affd700 1 -- 172.21.15.100:0/1397464668 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff78c04da50 con 0x7ff79415d5b0 2024-08-17T02:22:16.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.768+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7940643a0 con 0x7ff780060860 2024-08-17T02:22:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.788+0000 7ff77affd700 1 -- 172.21.15.100:0/1397464668 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264365 (secure 0 0 0) 0x7ff7940643a0 con 0x7ff780060860 2024-08-17T02:22:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:16.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff780060860 msgr2=0x7ff780062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff780060860 0x7ff780062d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7840091e0 tx=0x7ff784017000 comp rx=0 tx=0).stop 2024-08-17T02:22:16.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff79415d5b0 msgr2=0x7ff794150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff79415d5b0 0x7ff794150730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009fc0 tx=0x7ff78c00b040 comp rx=0 tx=0).stop 2024-08-17T02:22:16.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 shutdown_connections 2024-08-17T02:22:16.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff780060860 0x7ff780062d10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff79415d5b0 0x7ff794150730 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff794159040 0x7ff7940b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 --2- 172.21.15.100:0/1397464668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7940d0010 0x7ff7940b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 >> 172.21.15.100:0/1397464668 conn(0x7ff794068250 msgr2=0x7ff794154660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 shutdown_connections 2024-08-17T02:22:16.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.792+0000 7ff793fff700 1 -- 172.21.15.100:0/1397464668 wait complete. 2024-08-17T02:22:16.801 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:16.809 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-08-17T02:22:15.224721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382048,"kb_used_data":2984,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592672,"statfs":{"total":773094113280,"available":770654896128,"internally_reserved":0,"allocated":3055616,"data_stored":1812404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003598"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143110+0000","last_change":"2024-08-17T02:22:05.137118+0000","last_active":"2024-08-17T02:22:05.143110+0000","last_peered":"2024-08-17T02:22:05.143110+0000","last_clean":"2024-08-17T02:22:05.143110+0000","last_became_active":"2024-08-17T02:22:05.136784+0000","last_became_peered":"2024-08-17T02:22:05.136784+0000","last_unstale":"2024-08-17T02:22:05.143110+0000","last_undegraded":"2024-08-17T02:22:05.143110+0000","last_fullsized":"2024-08-17T02:22:05.143110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:04:25.260837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133391+0000","last_change":"2024-08-17T02:22:05.129636+0000","last_active":"2024-08-17T02:22:05.133391+0000","last_peered":"2024-08-17T02:22:05.133391+0000","last_clean":"2024-08-17T02:22:05.133391+0000","last_became_active":"2024-08-17T02:22:05.129342+0000","last_became_peered":"2024-08-17T02:22:05.129342+0000","last_unstale":"2024-08-17T02:22:05.133391+0000","last_undegraded":"2024-08-17T02:22:05.133391+0000","last_fullsized":"2024-08-17T02:22:05.133391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.131713+0000","last_change":"2024-08-17T02:22:05.128097+0000","last_active":"2024-08-17T02:22:05.131713+0000","last_peered":"2024-08-17T02:22:05.131713+0000","last_clean":"2024-08-17T02:22:05.131713+0000","last_became_active":"2024-08-17T02:22:05.127786+0000","last_became_peered":"2024-08-17T02:22:05.127786+0000","last_unstale":"2024-08-17T02:22:05.131713+0000","last_undegraded":"2024-08-17T02:22:05.131713+0000","last_fullsized":"2024-08-17T02:22:05.131713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:47.605286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142739+0000","last_change":"2024-08-17T02:22:05.138911+0000","last_active":"2024-08-17T02:22:05.142739+0000","last_peered":"2024-08-17T02:22:05.142739+0000","last_clean":"2024-08-17T02:22:05.142739+0000","last_became_active":"2024-08-17T02:22:05.134886+0000","last_became_peered":"2024-08-17T02:22:05.134886+0000","last_unstale":"2024-08-17T02:22:05.142739+0000","last_undegraded":"2024-08-17T02:22:05.142739+0000","last_fullsized":"2024-08-17T02:22:05.142739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:04:25.260837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143121+0000","last_change":"2024-08-17T02:22:05.137212+0000","last_active":"2024-08-17T02:22:05.143121+0000","last_peered":"2024-08-17T02:22:05.143121+0000","last_clean":"2024-08-17T02:22:05.143121+0000","last_became_active":"2024-08-17T02:22:05.136566+0000","last_became_peered":"2024-08-17T02:22:05.136566+0000","last_unstale":"2024-08-17T02:22:05.143121+0000","last_undegraded":"2024-08-17T02:22:05.143121+0000","last_fullsized":"2024-08-17T02:22:05.143121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:49.230407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141537+0000","last_change":"2024-08-17T02:22:05.130974+0000","last_active":"2024-08-17T02:22:05.141537+0000","last_peered":"2024-08-17T02:22:05.141537+0000","last_clean":"2024-08-17T02:22:05.141537+0000","last_became_active":"2024-08-17T02:22:05.130696+0000","last_became_peered":"2024-08-17T02:22:05.130696+0000","last_unstale":"2024-08-17T02:22:05.141537+0000","last_undegraded":"2024-08-17T02:22:05.141537+0000","last_fullsized":"2024-08-17T02:22:05.141537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:49.230407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143027+0000","last_change":"2024-08-17T02:22:05.137564+0000","last_active":"2024-08-17T02:22:05.143027+0000","last_peered":"2024-08-17T02:22:05.143027+0000","last_clean":"2024-08-17T02:22:05.143027+0000","last_became_active":"2024-08-17T02:22:05.137060+0000","last_became_peered":"2024-08-17T02:22:05.137060+0000","last_unstale":"2024-08-17T02:22:05.143027+0000","last_undegraded":"2024-08-17T02:22:05.143027+0000","last_fullsized":"2024-08-17T02:22:05.143027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:48:26.336166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146835+0000","last_change":"2024-08-17T02:22:05.143886+0000","last_active":"2024-08-17T02:22:05.146835+0000","last_peered":"2024-08-17T02:22:05.146835+0000","last_clean":"2024-08-17T02:22:05.146835+0000","last_became_active":"2024-08-17T02:22:05.143537+0000","last_became_peered":"2024-08-17T02:22:05.143537+0000","last_unstale":"2024-08-17T02:22:05.146835+0000","last_undegraded":"2024-08-17T02:22:05.146835+0000","last_fullsized":"2024-08-17T02:22:05.146835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:58:32.777645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146777+0000","last_change":"2024-08-17T02:22:05.142931+0000","last_active":"2024-08-17T02:22:05.146777+0000","last_peered":"2024-08-17T02:22:05.146777+0000","last_clean":"2024-08-17T02:22:05.146777+0000","last_became_active":"2024-08-17T02:22:05.142076+0000","last_became_peered":"2024-08-17T02:22:05.142076+0000","last_unstale":"2024-08-17T02:22:05.146777+0000","last_undegraded":"2024-08-17T02:22:05.146777+0000","last_fullsized":"2024-08-17T02:22:05.146777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:54:13.258567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133864+0000","last_change":"2024-08-17T02:22:05.131490+0000","last_active":"2024-08-17T02:22:05.133864+0000","last_peered":"2024-08-17T02:22:05.133864+0000","last_clean":"2024-08-17T02:22:05.133864+0000","last_became_active":"2024-08-17T02:22:05.131184+0000","last_became_peered":"2024-08-17T02:22:05.131184+0000","last_unstale":"2024-08-17T02:22:05.133864+0000","last_undegraded":"2024-08-17T02:22:05.133864+0000","last_fullsized":"2024-08-17T02:22:05.133864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134444+0000","last_change":"2024-08-17T02:22:05.131423+0000","last_active":"2024-08-17T02:22:05.134444+0000","last_peered":"2024-08-17T02:22:05.134444+0000","last_clean":"2024-08-17T02:22:05.134444+0000","last_became_active":"2024-08-17T02:22:05.131081+0000","last_became_peered":"2024-08-17T02:22:05.131081+0000","last_unstale":"2024-08-17T02:22:05.134444+0000","last_undegraded":"2024-08-17T02:22:05.134444+0000","last_fullsized":"2024-08-17T02:22:05.134444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133327+0000","last_change":"2024-08-17T02:22:05.129487+0000","last_active":"2024-08-17T02:22:05.133327+0000","last_peered":"2024-08-17T02:22:05.133327+0000","last_clean":"2024-08-17T02:22:05.133327+0000","last_became_active":"2024-08-17T02:22:05.129222+0000","last_became_peered":"2024-08-17T02:22:05.129222+0000","last_unstale":"2024-08-17T02:22:05.133327+0000","last_undegraded":"2024-08-17T02:22:05.133327+0000","last_fullsized":"2024-08-17T02:22:05.133327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:02.706489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138694+0000","last_change":"2024-08-17T02:22:05.134693+0000","last_active":"2024-08-17T02:22:05.138694+0000","last_peered":"2024-08-17T02:22:05.138694+0000","last_clean":"2024-08-17T02:22:05.138694+0000","last_became_active":"2024-08-17T02:22:05.134300+0000","last_became_peered":"2024-08-17T02:22:05.134300+0000","last_unstale":"2024-08-17T02:22:05.138694+0000","last_undegraded":"2024-08-17T02:22:05.138694+0000","last_fullsized":"2024-08-17T02:22:05.138694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:37.064590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.131330+0000","last_change":"2024-08-17T02:22:05.128086+0000","last_active":"2024-08-17T02:22:05.131330+0000","last_peered":"2024-08-17T02:22:05.131330+0000","last_clean":"2024-08-17T02:22:05.131330+0000","last_became_active":"2024-08-17T02:22:05.127750+0000","last_became_peered":"2024-08-17T02:22:05.127750+0000","last_unstale":"2024-08-17T02:22:05.131330+0000","last_undegraded":"2024-08-17T02:22:05.131330+0000","last_fullsized":"2024-08-17T02:22:05.131330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:01:27.557725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.123798+0000","last_change":"2024-08-17T02:22:05.123028+0000","last_active":"2024-08-17T02:22:05.123798+0000","last_peered":"2024-08-17T02:22:05.123798+0000","last_clean":"2024-08-17T02:22:05.123798+0000","last_became_active":"2024-08-17T02:22:05.122711+0000","last_became_peered":"2024-08-17T02:22:05.122711+0000","last_unstale":"2024-08-17T02:22:05.123798+0000","last_undegraded":"2024-08-17T02:22:05.123798+0000","last_fullsized":"2024-08-17T02:22:05.123798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141452+0000","last_change":"2024-08-17T02:22:05.130380+0000","last_active":"2024-08-17T02:22:05.141452+0000","last_peered":"2024-08-17T02:22:05.141452+0000","last_clean":"2024-08-17T02:22:05.141452+0000","last_became_active":"2024-08-17T02:22:05.130077+0000","last_became_peered":"2024-08-17T02:22:05.130077+0000","last_unstale":"2024-08-17T02:22:05.141452+0000","last_undegraded":"2024-08-17T02:22:05.141452+0000","last_fullsized":"2024-08-17T02:22:05.141452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:27:57.075296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142733+0000","last_change":"2024-08-17T02:22:05.139113+0000","last_active":"2024-08-17T02:22:05.142733+0000","last_peered":"2024-08-17T02:22:05.142733+0000","last_clean":"2024-08-17T02:22:05.142733+0000","last_became_active":"2024-08-17T02:22:05.138752+0000","last_became_peered":"2024-08-17T02:22:05.138752+0000","last_unstale":"2024-08-17T02:22:05.142733+0000","last_undegraded":"2024-08-17T02:22:05.142733+0000","last_fullsized":"2024-08-17T02:22:05.142733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:21:52.626562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.129913+0000","last_change":"2024-08-17T02:22:05.125930+0000","last_active":"2024-08-17T02:22:05.129913+0000","last_peered":"2024-08-17T02:22:05.129913+0000","last_clean":"2024-08-17T02:22:05.129913+0000","last_became_active":"2024-08-17T02:22:05.125640+0000","last_became_peered":"2024-08-17T02:22:05.125640+0000","last_unstale":"2024-08-17T02:22:05.129913+0000","last_undegraded":"2024-08-17T02:22:05.129913+0000","last_fullsized":"2024-08-17T02:22:05.129913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:59:54.522675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142959+0000","last_change":"2024-08-17T02:22:05.137818+0000","last_active":"2024-08-17T02:22:05.142959+0000","last_peered":"2024-08-17T02:22:05.142959+0000","last_clean":"2024-08-17T02:22:05.142959+0000","last_became_active":"2024-08-17T02:22:05.137531+0000","last_became_peered":"2024-08-17T02:22:05.137531+0000","last_unstale":"2024-08-17T02:22:05.142959+0000","last_undegraded":"2024-08-17T02:22:05.142959+0000","last_fullsized":"2024-08-17T02:22:05.142959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:44:40.942386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.137413+0000","last_change":"2024-08-17T02:22:05.135163+0000","last_active":"2024-08-17T02:22:05.137413+0000","last_peered":"2024-08-17T02:22:05.137413+0000","last_clean":"2024-08-17T02:22:05.137413+0000","last_became_active":"2024-08-17T02:22:05.134647+0000","last_became_peered":"2024-08-17T02:22:05.134647+0000","last_unstale":"2024-08-17T02:22:05.137413+0000","last_undegraded":"2024-08-17T02:22:05.137413+0000","last_fullsized":"2024-08-17T02:22:05.137413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:21.690535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144205+0000","last_change":"2024-08-17T02:22:05.137440+0000","last_active":"2024-08-17T02:22:05.144205+0000","last_peered":"2024-08-17T02:22:05.144205+0000","last_clean":"2024-08-17T02:22:05.144205+0000","last_became_active":"2024-08-17T02:22:05.137081+0000","last_became_peered":"2024-08-17T02:22:05.137081+0000","last_unstale":"2024-08-17T02:22:05.144205+0000","last_undegraded":"2024-08-17T02:22:05.144205+0000","last_fullsized":"2024-08-17T02:22:05.144205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:34:43.102929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142978+0000","last_change":"2024-08-17T02:22:05.139220+0000","last_active":"2024-08-17T02:22:05.142978+0000","last_peered":"2024-08-17T02:22:05.142978+0000","last_clean":"2024-08-17T02:22:05.142978+0000","last_became_active":"2024-08-17T02:22:05.138838+0000","last_became_peered":"2024-08-17T02:22:05.138838+0000","last_unstale":"2024-08-17T02:22:05.142978+0000","last_undegraded":"2024-08-17T02:22:05.142978+0000","last_fullsized":"2024-08-17T02:22:05.142978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:47:26.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135504+0000","last_change":"2024-08-17T02:22:05.132463+0000","last_active":"2024-08-17T02:22:05.135504+0000","last_peered":"2024-08-17T02:22:05.135504+0000","last_clean":"2024-08-17T02:22:05.135504+0000","last_became_active":"2024-08-17T02:22:05.131741+0000","last_became_peered":"2024-08-17T02:22:05.131741+0000","last_unstale":"2024-08-17T02:22:05.135504+0000","last_undegraded":"2024-08-17T02:22:05.135504+0000","last_fullsized":"2024-08-17T02:22:05.135504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142777+0000","last_change":"2024-08-17T02:22:05.140510+0000","last_active":"2024-08-17T02:22:05.142777+0000","last_peered":"2024-08-17T02:22:05.142777+0000","last_clean":"2024-08-17T02:22:05.142777+0000","last_became_active":"2024-08-17T02:22:05.139145+0000","last_became_peered":"2024-08-17T02:22:05.139145+0000","last_unstale":"2024-08-17T02:22:05.142777+0000","last_undegraded":"2024-08-17T02:22:05.142777+0000","last_fullsized":"2024-08-17T02:22:05.142777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:58:53.092915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135524+0000","last_change":"2024-08-17T02:22:05.133736+0000","last_active":"2024-08-17T02:22:05.135524+0000","last_peered":"2024-08-17T02:22:05.135524+0000","last_clean":"2024-08-17T02:22:05.135524+0000","last_became_active":"2024-08-17T02:22:05.132607+0000","last_became_peered":"2024-08-17T02:22:05.132607+0000","last_unstale":"2024-08-17T02:22:05.135524+0000","last_undegraded":"2024-08-17T02:22:05.135524+0000","last_fullsized":"2024-08-17T02:22:05.135524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141845+0000","last_change":"2024-08-17T02:22:05.137747+0000","last_active":"2024-08-17T02:22:05.141845+0000","last_peered":"2024-08-17T02:22:05.141845+0000","last_clean":"2024-08-17T02:22:05.141845+0000","last_became_active":"2024-08-17T02:22:05.137464+0000","last_became_peered":"2024-08-17T02:22:05.137464+0000","last_unstale":"2024-08-17T02:22:05.141845+0000","last_undegraded":"2024-08-17T02:22:05.141845+0000","last_fullsized":"2024-08-17T02:22:05.141845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138959+0000","last_change":"2024-08-17T02:22:05.133235+0000","last_active":"2024-08-17T02:22:05.138959+0000","last_peered":"2024-08-17T02:22:05.138959+0000","last_clean":"2024-08-17T02:22:05.138959+0000","last_became_active":"2024-08-17T02:22:05.132962+0000","last_became_peered":"2024-08-17T02:22:05.132962+0000","last_unstale":"2024-08-17T02:22:05.138959+0000","last_undegraded":"2024-08-17T02:22:05.138959+0000","last_fullsized":"2024-08-17T02:22:05.138959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135434+0000","last_change":"2024-08-17T02:22:05.132315+0000","last_active":"2024-08-17T02:22:05.135434+0000","last_peered":"2024-08-17T02:22:05.135434+0000","last_clean":"2024-08-17T02:22:05.135434+0000","last_became_active":"2024-08-17T02:22:05.131822+0000","last_became_peered":"2024-08-17T02:22:05.131822+0000","last_unstale":"2024-08-17T02:22:05.135434+0000","last_undegraded":"2024-08-17T02:22:05.135434+0000","last_fullsized":"2024-08-17T02:22:05.135434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139793+0000","last_change":"2024-08-17T02:22:05.136258+0000","last_active":"2024-08-17T02:22:05.139793+0000","last_peered":"2024-08-17T02:22:05.139793+0000","last_clean":"2024-08-17T02:22:05.139793+0000","last_became_active":"2024-08-17T02:22:05.135573+0000","last_became_peered":"2024-08-17T02:22:05.135573+0000","last_unstale":"2024-08-17T02:22:05.139793+0000","last_undegraded":"2024-08-17T02:22:05.139793+0000","last_fullsized":"2024-08-17T02:22:05.139793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:36:55.030030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142962+0000","last_change":"2024-08-17T02:22:05.139131+0000","last_active":"2024-08-17T02:22:05.142962+0000","last_peered":"2024-08-17T02:22:05.142962+0000","last_clean":"2024-08-17T02:22:05.142962+0000","last_became_active":"2024-08-17T02:22:05.138484+0000","last_became_peered":"2024-08-17T02:22:05.138484+0000","last_unstale":"2024-08-17T02:22:05.142962+0000","last_undegraded":"2024-08-17T02:22:05.142962+0000","last_fullsized":"2024-08-17T02:22:05.142962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:49:31.875303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134588+0000","last_change":"2024-08-17T02:22:05.128466+0000","last_active":"2024-08-17T02:22:05.134588+0000","last_peered":"2024-08-17T02:22:05.134588+0000","last_clean":"2024-08-17T02:22:05.134588+0000","last_became_active":"2024-08-17T02:22:05.128215+0000","last_became_peered":"2024-08-17T02:22:05.128215+0000","last_unstale":"2024-08-17T02:22:05.134588+0000","last_undegraded":"2024-08-17T02:22:05.134588+0000","last_fullsized":"2024-08-17T02:22:05.134588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:49.930124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133891+0000","last_change":"2024-08-17T02:22:05.131735+0000","last_active":"2024-08-17T02:22:05.133891+0000","last_peered":"2024-08-17T02:22:05.133891+0000","last_clean":"2024-08-17T02:22:05.133891+0000","last_became_active":"2024-08-17T02:22:05.131343+0000","last_became_peered":"2024-08-17T02:22:05.131343+0000","last_unstale":"2024-08-17T02:22:05.133891+0000","last_undegraded":"2024-08-17T02:22:05.133891+0000","last_fullsized":"2024-08-17T02:22:05.133891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:41:42.553817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.128947+0000","last_change":"2024-08-17T02:22:05.124427+0000","last_active":"2024-08-17T02:22:05.128947+0000","last_peered":"2024-08-17T02:22:05.128947+0000","last_clean":"2024-08-17T02:22:05.128947+0000","last_became_active":"2024-08-17T02:22:05.124149+0000","last_became_peered":"2024-08-17T02:22:05.124149+0000","last_unstale":"2024-08-17T02:22:05.128947+0000","last_undegraded":"2024-08-17T02:22:05.128947+0000","last_fullsized":"2024-08-17T02:22:05.128947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:57.001696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.140231+0000","last_change":"2024-08-17T02:22:04.140654+0000","last_active":"2024-08-17T02:22:05.140231+0000","last_peered":"2024-08-17T02:22:05.140231+0000","last_clean":"2024-08-17T02:22:05.140231+0000","last_became_active":"2024-08-17T02:22:04.140316+0000","last_became_peered":"2024-08-17T02:22:04.140316+0000","last_unstale":"2024-08-17T02:22:05.140231+0000","last_undegraded":"2024-08-17T02:22:05.140231+0000","last_fullsized":"2024-08-17T02:22:05.140231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136973+0000","last_change":"2024-08-17T02:22:05.130524+0000","last_active":"2024-08-17T02:22:05.136973+0000","last_peered":"2024-08-17T02:22:05.136973+0000","last_clean":"2024-08-17T02:22:05.136973+0000","last_became_active":"2024-08-17T02:22:05.130188+0000","last_became_peered":"2024-08-17T02:22:05.130188+0000","last_unstale":"2024-08-17T02:22:05.136973+0000","last_undegraded":"2024-08-17T02:22:05.136973+0000","last_fullsized":"2024-08-17T02:22:05.136973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:11:14.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134335+0000","last_change":"2024-08-17T02:22:04.123306+0000","last_active":"2024-08-17T02:22:05.134335+0000","last_peered":"2024-08-17T02:22:05.134335+0000","last_clean":"2024-08-17T02:22:05.134335+0000","last_became_active":"2024-08-17T02:22:04.122968+0000","last_became_peered":"2024-08-17T02:22:04.122968+0000","last_unstale":"2024-08-17T02:22:05.134335+0000","last_undegraded":"2024-08-17T02:22:05.134335+0000","last_fullsized":"2024-08-17T02:22:05.134335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:26:59.050552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.124555+0000","last_change":"2024-08-17T02:22:04.124101+0000","last_active":"2024-08-17T02:22:05.124555+0000","last_peered":"2024-08-17T02:22:05.124555+0000","last_clean":"2024-08-17T02:22:05.124555+0000","last_became_active":"2024-08-17T02:22:04.123756+0000","last_became_peered":"2024-08-17T02:22:04.123756+0000","last_unstale":"2024-08-17T02:22:05.124555+0000","last_undegraded":"2024-08-17T02:22:05.124555+0000","last_fullsized":"2024-08-17T02:22:05.124555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:34:34.776460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135564+0000","last_change":"2024-08-17T02:22:05.133677+0000","last_active":"2024-08-17T02:22:05.135564+0000","last_peered":"2024-08-17T02:22:05.135564+0000","last_clean":"2024-08-17T02:22:05.135564+0000","last_became_active":"2024-08-17T02:22:05.132419+0000","last_became_peered":"2024-08-17T02:22:05.132419+0000","last_unstale":"2024-08-17T02:22:05.135564+0000","last_undegraded":"2024-08-17T02:22:05.135564+0000","last_fullsized":"2024-08-17T02:22:05.135564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:00:22.156028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.115878+0000","last_change":"2024-08-17T02:22:04.129312+0000","last_active":"2024-08-17T02:22:05.115878+0000","last_peered":"2024-08-17T02:22:05.115878+0000","last_clean":"2024-08-17T02:22:05.115878+0000","last_became_active":"2024-08-17T02:22:04.129003+0000","last_became_peered":"2024-08-17T02:22:04.129003+0000","last_unstale":"2024-08-17T02:22:05.115878+0000","last_undegraded":"2024-08-17T02:22:05.115878+0000","last_fullsized":"2024-08-17T02:22:05.115878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:48:19.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139117+0000","last_change":"2024-08-17T02:22:05.133370+0000","last_active":"2024-08-17T02:22:05.139117+0000","last_peered":"2024-08-17T02:22:05.139117+0000","last_clean":"2024-08-17T02:22:05.139117+0000","last_became_active":"2024-08-17T02:22:05.133041+0000","last_became_peered":"2024-08-17T02:22:05.133041+0000","last_unstale":"2024-08-17T02:22:05.139117+0000","last_undegraded":"2024-08-17T02:22:05.139117+0000","last_fullsized":"2024-08-17T02:22:05.139117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:50:04.925615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120010+0000","last_change":"2024-08-17T02:22:04.134772+0000","last_active":"2024-08-17T02:22:05.120010+0000","last_peered":"2024-08-17T02:22:05.120010+0000","last_clean":"2024-08-17T02:22:05.120010+0000","last_became_active":"2024-08-17T02:22:04.134509+0000","last_became_peered":"2024-08-17T02:22:04.134509+0000","last_unstale":"2024-08-17T02:22:05.120010+0000","last_undegraded":"2024-08-17T02:22:05.120010+0000","last_fullsized":"2024-08-17T02:22:05.120010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:01.702706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142478+0000","last_change":"2024-08-17T02:22:05.139006+0000","last_active":"2024-08-17T02:22:05.142478+0000","last_peered":"2024-08-17T02:22:05.142478+0000","last_clean":"2024-08-17T02:22:05.142478+0000","last_became_active":"2024-08-17T02:22:05.138609+0000","last_became_peered":"2024-08-17T02:22:05.138609+0000","last_unstale":"2024-08-17T02:22:05.142478+0000","last_undegraded":"2024-08-17T02:22:05.142478+0000","last_fullsized":"2024-08-17T02:22:05.142478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.116705+0000","last_change":"2024-08-17T02:22:04.129251+0000","last_active":"2024-08-17T02:22:05.116705+0000","last_peered":"2024-08-17T02:22:05.116705+0000","last_clean":"2024-08-17T02:22:05.116705+0000","last_became_active":"2024-08-17T02:22:04.126603+0000","last_became_peered":"2024-08-17T02:22:04.126603+0000","last_unstale":"2024-08-17T02:22:05.116705+0000","last_undegraded":"2024-08-17T02:22:05.116705+0000","last_fullsized":"2024-08-17T02:22:05.116705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139326+0000","last_change":"2024-08-17T02:22:05.131334+0000","last_active":"2024-08-17T02:22:05.139326+0000","last_peered":"2024-08-17T02:22:05.139326+0000","last_clean":"2024-08-17T02:22:05.139326+0000","last_became_active":"2024-08-17T02:22:05.130325+0000","last_became_peered":"2024-08-17T02:22:05.130325+0000","last_unstale":"2024-08-17T02:22:05.139326+0000","last_undegraded":"2024-08-17T02:22:05.139326+0000","last_fullsized":"2024-08-17T02:22:05.139326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:47:26.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142458+0000","last_change":"2024-08-17T02:22:04.135036+0000","last_active":"2024-08-17T02:22:05.142458+0000","last_peered":"2024-08-17T02:22:05.142458+0000","last_clean":"2024-08-17T02:22:05.142458+0000","last_became_active":"2024-08-17T02:22:04.134786+0000","last_became_peered":"2024-08-17T02:22:04.134786+0000","last_unstale":"2024-08-17T02:22:05.142458+0000","last_undegraded":"2024-08-17T02:22:05.142458+0000","last_fullsized":"2024-08-17T02:22:05.142458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.997912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146826+0000","last_change":"2024-08-17T02:22:05.141338+0000","last_active":"2024-08-17T02:22:05.146826+0000","last_peered":"2024-08-17T02:22:05.146826+0000","last_clean":"2024-08-17T02:22:05.146826+0000","last_became_active":"2024-08-17T02:22:05.139395+0000","last_became_peered":"2024-08-17T02:22:05.139395+0000","last_unstale":"2024-08-17T02:22:05.146826+0000","last_undegraded":"2024-08-17T02:22:05.146826+0000","last_fullsized":"2024-08-17T02:22:05.146826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:07.057717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146729+0000","last_change":"2024-08-17T02:22:04.127992+0000","last_active":"2024-08-17T02:22:05.146729+0000","last_peered":"2024-08-17T02:22:05.146729+0000","last_clean":"2024-08-17T02:22:05.146729+0000","last_became_active":"2024-08-17T02:22:04.127656+0000","last_became_peered":"2024-08-17T02:22:04.127656+0000","last_unstale":"2024-08-17T02:22:05.146729+0000","last_undegraded":"2024-08-17T02:22:05.146729+0000","last_fullsized":"2024-08-17T02:22:05.146729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:06.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142876+0000","last_change":"2024-08-17T02:22:05.136992+0000","last_active":"2024-08-17T02:22:05.142876+0000","last_peered":"2024-08-17T02:22:05.142876+0000","last_clean":"2024-08-17T02:22:05.142876+0000","last_became_active":"2024-08-17T02:22:05.136611+0000","last_became_peered":"2024-08-17T02:22:05.136611+0000","last_unstale":"2024-08-17T02:22:05.142876+0000","last_undegraded":"2024-08-17T02:22:05.142876+0000","last_fullsized":"2024-08-17T02:22:05.142876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:27:57.075296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147451+0000","last_change":"2024-08-17T02:22:04.136005+0000","last_active":"2024-08-17T02:22:05.147451+0000","last_peered":"2024-08-17T02:22:05.147451+0000","last_clean":"2024-08-17T02:22:05.147451+0000","last_became_active":"2024-08-17T02:22:04.135718+0000","last_became_peered":"2024-08-17T02:22:04.135718+0000","last_unstale":"2024-08-17T02:22:05.147451+0000","last_undegraded":"2024-08-17T02:22:05.147451+0000","last_fullsized":"2024-08-17T02:22:05.147451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:11.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144007+0000","last_change":"2024-08-17T02:22:05.137445+0000","last_active":"2024-08-17T02:22:05.144007+0000","last_peered":"2024-08-17T02:22:05.144007+0000","last_clean":"2024-08-17T02:22:05.144007+0000","last_became_active":"2024-08-17T02:22:05.137047+0000","last_became_peered":"2024-08-17T02:22:05.137047+0000","last_unstale":"2024-08-17T02:22:05.144007+0000","last_undegraded":"2024-08-17T02:22:05.144007+0000","last_fullsized":"2024-08-17T02:22:05.144007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:49.930124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.117051+0000","last_change":"2024-08-17T02:22:04.128882+0000","last_active":"2024-08-17T02:22:05.117051+0000","last_peered":"2024-08-17T02:22:05.117051+0000","last_clean":"2024-08-17T02:22:05.117051+0000","last_became_active":"2024-08-17T02:22:04.124521+0000","last_became_peered":"2024-08-17T02:22:04.124521+0000","last_unstale":"2024-08-17T02:22:05.117051+0000","last_undegraded":"2024-08-17T02:22:05.117051+0000","last_fullsized":"2024-08-17T02:22:05.117051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146565+0000","last_change":"2024-08-17T02:22:05.140259+0000","last_active":"2024-08-17T02:22:05.146565+0000","last_peered":"2024-08-17T02:22:05.146565+0000","last_clean":"2024-08-17T02:22:05.146565+0000","last_became_active":"2024-08-17T02:22:05.139966+0000","last_became_peered":"2024-08-17T02:22:05.139966+0000","last_unstale":"2024-08-17T02:22:05.146565+0000","last_undegraded":"2024-08-17T02:22:05.146565+0000","last_fullsized":"2024-08-17T02:22:05.146565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:01:27.557725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120796+0000","last_change":"2024-08-17T02:22:04.122812+0000","last_active":"2024-08-17T02:22:05.120796+0000","last_peered":"2024-08-17T02:22:05.120796+0000","last_clean":"2024-08-17T02:22:05.120796+0000","last_became_active":"2024-08-17T02:22:04.122506+0000","last_became_peered":"2024-08-17T02:22:04.122506+0000","last_unstale":"2024-08-17T02:22:05.120796+0000","last_undegraded":"2024-08-17T02:22:05.120796+0000","last_fullsized":"2024-08-17T02:22:05.120796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:56.324601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133838+0000","last_change":"2024-08-17T02:22:05.131385+0000","last_active":"2024-08-17T02:22:05.133838+0000","last_peered":"2024-08-17T02:22:05.133838+0000","last_clean":"2024-08-17T02:22:05.133838+0000","last_became_active":"2024-08-17T02:22:05.130986+0000","last_became_peered":"2024-08-17T02:22:05.130986+0000","last_unstale":"2024-08-17T02:22:05.133838+0000","last_undegraded":"2024-08-17T02:22:05.133838+0000","last_fullsized":"2024-08-17T02:22:05.133838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.132407+0000","last_change":"2024-08-17T02:22:04.125802+0000","last_active":"2024-08-17T02:22:05.132407+0000","last_peered":"2024-08-17T02:22:05.132407+0000","last_clean":"2024-08-17T02:22:05.132407+0000","last_became_active":"2024-08-17T02:22:04.125462+0000","last_became_peered":"2024-08-17T02:22:04.125462+0000","last_unstale":"2024-08-17T02:22:05.132407+0000","last_undegraded":"2024-08-17T02:22:05.132407+0000","last_fullsized":"2024-08-17T02:22:05.132407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.143008+0000","last_change":"2024-08-17T02:22:05.137082+0000","last_active":"2024-08-17T02:22:05.143008+0000","last_peered":"2024-08-17T02:22:05.143008+0000","last_clean":"2024-08-17T02:22:05.143008+0000","last_became_active":"2024-08-17T02:22:05.136773+0000","last_became_peered":"2024-08-17T02:22:05.136773+0000","last_unstale":"2024-08-17T02:22:05.143008+0000","last_undegraded":"2024-08-17T02:22:05.143008+0000","last_fullsized":"2024-08-17T02:22:05.143008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142302+0000","last_change":"2024-08-17T02:22:04.136447+0000","last_active":"2024-08-17T02:22:05.142302+0000","last_peered":"2024-08-17T02:22:05.142302+0000","last_clean":"2024-08-17T02:22:05.142302+0000","last_became_active":"2024-08-17T02:22:04.136105+0000","last_became_peered":"2024-08-17T02:22:04.136105+0000","last_unstale":"2024-08-17T02:22:05.142302+0000","last_undegraded":"2024-08-17T02:22:05.142302+0000","last_fullsized":"2024-08-17T02:22:05.142302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:29:10.820076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147858+0000","last_change":"2024-08-17T02:22:05.145138+0000","last_active":"2024-08-17T02:22:05.147858+0000","last_peered":"2024-08-17T02:22:05.147858+0000","last_clean":"2024-08-17T02:22:05.147858+0000","last_became_active":"2024-08-17T02:22:05.144754+0000","last_became_peered":"2024-08-17T02:22:05.144754+0000","last_unstale":"2024-08-17T02:22:05.147858+0000","last_undegraded":"2024-08-17T02:22:05.147858+0000","last_fullsized":"2024-08-17T02:22:05.147858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:36:55.030030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133637+0000","last_change":"2024-08-17T02:22:04.128289+0000","last_active":"2024-08-17T02:22:05.133637+0000","last_peered":"2024-08-17T02:22:05.133637+0000","last_clean":"2024-08-17T02:22:05.133637+0000","last_became_active":"2024-08-17T02:22:04.127984+0000","last_became_peered":"2024-08-17T02:22:04.127984+0000","last_unstale":"2024-08-17T02:22:05.133637+0000","last_undegraded":"2024-08-17T02:22:05.133637+0000","last_fullsized":"2024-08-17T02:22:05.133637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.147852+0000","last_change":"2024-08-17T02:22:05.145108+0000","last_active":"2024-08-17T02:22:05.147852+0000","last_peered":"2024-08-17T02:22:05.147852+0000","last_clean":"2024-08-17T02:22:05.147852+0000","last_became_active":"2024-08-17T02:22:05.144771+0000","last_became_peered":"2024-08-17T02:22:05.144771+0000","last_unstale":"2024-08-17T02:22:05.147852+0000","last_undegraded":"2024-08-17T02:22:05.147852+0000","last_fullsized":"2024-08-17T02:22:05.147852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:34:43.102929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139054+0000","last_change":"2024-08-17T02:22:04.125374+0000","last_active":"2024-08-17T02:22:05.139054+0000","last_peered":"2024-08-17T02:22:05.139054+0000","last_clean":"2024-08-17T02:22:05.139054+0000","last_became_active":"2024-08-17T02:22:04.125059+0000","last_became_peered":"2024-08-17T02:22:04.125059+0000","last_unstale":"2024-08-17T02:22:05.139054+0000","last_undegraded":"2024-08-17T02:22:05.139054+0000","last_fullsized":"2024-08-17T02:22:05.139054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:31:01.702706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.142758+0000","last_change":"2024-08-17T02:22:05.139051+0000","last_active":"2024-08-17T02:22:05.142758+0000","last_peered":"2024-08-17T02:22:05.142758+0000","last_clean":"2024-08-17T02:22:05.142758+0000","last_became_active":"2024-08-17T02:22:05.135262+0000","last_became_peered":"2024-08-17T02:22:05.135262+0000","last_unstale":"2024-08-17T02:22:05.142758+0000","last_undegraded":"2024-08-17T02:22:05.142758+0000","last_fullsized":"2024-08-17T02:22:05.142758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:42:24.349708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.118431+0000","last_change":"2024-08-17T02:22:04.137875+0000","last_active":"2024-08-17T02:22:05.118431+0000","last_peered":"2024-08-17T02:22:05.118431+0000","last_clean":"2024-08-17T02:22:05.118431+0000","last_became_active":"2024-08-17T02:22:04.137590+0000","last_became_peered":"2024-08-17T02:22:04.137590+0000","last_unstale":"2024-08-17T02:22:05.118431+0000","last_undegraded":"2024-08-17T02:22:05.118431+0000","last_fullsized":"2024-08-17T02:22:05.118431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146797+0000","last_change":"2024-08-17T02:22:05.142646+0000","last_active":"2024-08-17T02:22:05.146797+0000","last_peered":"2024-08-17T02:22:05.146797+0000","last_clean":"2024-08-17T02:22:05.146797+0000","last_became_active":"2024-08-17T02:22:05.141915+0000","last_became_peered":"2024-08-17T02:22:05.141915+0000","last_unstale":"2024-08-17T02:22:05.146797+0000","last_undegraded":"2024-08-17T02:22:05.146797+0000","last_fullsized":"2024-08-17T02:22:05.146797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:30:37.267266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120860+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:05.120860+0000","last_peered":"2024-08-17T02:22:05.120860+0000","last_clean":"2024-08-17T02:22:05.120860+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:05.120860+0000","last_undegraded":"2024-08-17T02:22:05.120860+0000","last_fullsized":"2024-08-17T02:22:05.120860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.116535+0000","last_change":"2024-08-17T02:22:04.129059+0000","last_active":"2024-08-17T02:22:05.116535+0000","last_peered":"2024-08-17T02:22:05.116535+0000","last_clean":"2024-08-17T02:22:05.116535+0000","last_became_active":"2024-08-17T02:22:04.126173+0000","last_became_peered":"2024-08-17T02:22:04.126173+0000","last_unstale":"2024-08-17T02:22:05.116535+0000","last_undegraded":"2024-08-17T02:22:05.116535+0000","last_fullsized":"2024-08-17T02:22:05.116535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.038634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.145732+0000","last_change":"2024-08-17T02:22:05.142911+0000","last_active":"2024-08-17T02:22:05.145732+0000","last_peered":"2024-08-17T02:22:05.145732+0000","last_clean":"2024-08-17T02:22:05.145732+0000","last_became_active":"2024-08-17T02:22:05.142665+0000","last_became_peered":"2024-08-17T02:22:05.142665+0000","last_unstale":"2024-08-17T02:22:05.145732+0000","last_undegraded":"2024-08-17T02:22:05.145732+0000","last_fullsized":"2024-08-17T02:22:05.145732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:41:42.553817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120935+0000","last_change":"2024-08-17T02:22:04.125465+0000","last_active":"2024-08-17T02:22:05.120935+0000","last_peered":"2024-08-17T02:22:05.120935+0000","last_clean":"2024-08-17T02:22:05.120935+0000","last_became_active":"2024-08-17T02:22:04.125205+0000","last_became_peered":"2024-08-17T02:22:04.125205+0000","last_unstale":"2024-08-17T02:22:05.120935+0000","last_undegraded":"2024-08-17T02:22:05.120935+0000","last_fullsized":"2024-08-17T02:22:05.120935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:59:53.518892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139508+0000","last_change":"2024-08-17T02:22:05.132007+0000","last_active":"2024-08-17T02:22:05.139508+0000","last_peered":"2024-08-17T02:22:05.139508+0000","last_clean":"2024-08-17T02:22:05.139508+0000","last_became_active":"2024-08-17T02:22:05.131083+0000","last_became_peered":"2024-08-17T02:22:05.131083+0000","last_unstale":"2024-08-17T02:22:05.139508+0000","last_undegraded":"2024-08-17T02:22:05.139508+0000","last_fullsized":"2024-08-17T02:22:05.139508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:15.303455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.119774+0000","last_change":"2024-08-17T02:22:04.128675+0000","last_active":"2024-08-17T02:22:05.119774+0000","last_peered":"2024-08-17T02:22:05.119774+0000","last_clean":"2024-08-17T02:22:05.119774+0000","last_became_active":"2024-08-17T02:22:04.128264+0000","last_became_peered":"2024-08-17T02:22:04.128264+0000","last_unstale":"2024-08-17T02:22:05.119774+0000","last_undegraded":"2024-08-17T02:22:05.119774+0000","last_fullsized":"2024-08-17T02:22:05.119774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.125552+0000","last_change":"2024-08-17T02:22:05.123180+0000","last_active":"2024-08-17T02:22:05.125552+0000","last_peered":"2024-08-17T02:22:05.125552+0000","last_clean":"2024-08-17T02:22:05.125552+0000","last_became_active":"2024-08-17T02:22:05.122887+0000","last_became_peered":"2024-08-17T02:22:05.122887+0000","last_unstale":"2024-08-17T02:22:05.125552+0000","last_undegraded":"2024-08-17T02:22:05.125552+0000","last_fullsized":"2024-08-17T02:22:05.125552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:55.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.117161+0000","last_change":"2024-08-17T02:22:04.125809+0000","last_active":"2024-08-17T02:22:05.117161+0000","last_peered":"2024-08-17T02:22:05.117161+0000","last_clean":"2024-08-17T02:22:05.117161+0000","last_became_active":"2024-08-17T02:22:04.125484+0000","last_became_peered":"2024-08-17T02:22:04.125484+0000","last_unstale":"2024-08-17T02:22:05.117161+0000","last_undegraded":"2024-08-17T02:22:05.117161+0000","last_fullsized":"2024-08-17T02:22:05.117161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.146600+0000","last_change":"2024-08-17T02:22:05.138522+0000","last_active":"2024-08-17T02:22:05.146600+0000","last_peered":"2024-08-17T02:22:05.146600+0000","last_clean":"2024-08-17T02:22:05.146600+0000","last_became_active":"2024-08-17T02:22:05.136133+0000","last_became_peered":"2024-08-17T02:22:05.136133+0000","last_unstale":"2024-08-17T02:22:05.146600+0000","last_undegraded":"2024-08-17T02:22:05.146600+0000","last_fullsized":"2024-08-17T02:22:05.146600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:48.803891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110226+0000","last_change":"2024-08-17T02:22:04.139328+0000","last_active":"2024-08-17T02:22:05.110226+0000","last_peered":"2024-08-17T02:22:05.110226+0000","last_clean":"2024-08-17T02:22:05.110226+0000","last_became_active":"2024-08-17T02:22:04.139078+0000","last_became_peered":"2024-08-17T02:22:04.139078+0000","last_unstale":"2024-08-17T02:22:05.110226+0000","last_undegraded":"2024-08-17T02:22:05.110226+0000","last_fullsized":"2024-08-17T02:22:05.110226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:26:59.050552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141791+0000","last_change":"2024-08-17T02:22:05.135288+0000","last_active":"2024-08-17T02:22:05.141791+0000","last_peered":"2024-08-17T02:22:05.141791+0000","last_clean":"2024-08-17T02:22:05.141791+0000","last_became_active":"2024-08-17T02:22:05.134902+0000","last_became_peered":"2024-08-17T02:22:05.134902+0000","last_unstale":"2024-08-17T02:22:05.141791+0000","last_undegraded":"2024-08-17T02:22:05.141791+0000","last_fullsized":"2024-08-17T02:22:05.141791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:41:22.363365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.127565+0000","last_change":"2024-08-17T02:22:04.136037+0000","last_active":"2024-08-17T02:22:05.127565+0000","last_peered":"2024-08-17T02:22:05.127565+0000","last_clean":"2024-08-17T02:22:05.127565+0000","last_became_active":"2024-08-17T02:22:04.135795+0000","last_became_peered":"2024-08-17T02:22:04.135795+0000","last_unstale":"2024-08-17T02:22:05.127565+0000","last_undegraded":"2024-08-17T02:22:05.127565+0000","last_fullsized":"2024-08-17T02:22:05.127565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:59.073112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.124422+0000","last_change":"2024-08-17T02:22:05.123134+0000","last_active":"2024-08-17T02:22:05.124422+0000","last_peered":"2024-08-17T02:22:05.124422+0000","last_clean":"2024-08-17T02:22:05.124422+0000","last_became_active":"2024-08-17T02:22:05.122849+0000","last_became_peered":"2024-08-17T02:22:05.122849+0000","last_unstale":"2024-08-17T02:22:05.124422+0000","last_undegraded":"2024-08-17T02:22:05.124422+0000","last_fullsized":"2024-08-17T02:22:05.124422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:37.064590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-08-17T02:22:07.113057+0000","last_change":"2024-08-17T02:22:04.124830+0000","last_active":"2024-08-17T02:22:07.113057+0000","last_peered":"2024-08-17T02:22:07.113057+0000","last_clean":"2024-08-17T02:22:07.113057+0000","last_became_active":"2024-08-17T02:22:04.124412+0000","last_became_peered":"2024-08-17T02:22:04.124412+0000","last_unstale":"2024-08-17T02:22:07.113057+0000","last_undegraded":"2024-08-17T02:22:07.113057+0000","last_fullsized":"2024-08-17T02:22:07.113057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:10:06.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.145579+0000","last_change":"2024-08-17T02:22:05.140560+0000","last_active":"2024-08-17T02:22:05.145579+0000","last_peered":"2024-08-17T02:22:05.145579+0000","last_clean":"2024-08-17T02:22:05.145579+0000","last_became_active":"2024-08-17T02:22:05.140249+0000","last_became_peered":"2024-08-17T02:22:05.140249+0000","last_unstale":"2024-08-17T02:22:05.145579+0000","last_undegraded":"2024-08-17T02:22:05.145579+0000","last_fullsized":"2024-08-17T02:22:05.145579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:24:46.183039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138031+0000","last_change":"2024-08-17T02:22:04.140288+0000","last_active":"2024-08-17T02:22:05.138031+0000","last_peered":"2024-08-17T02:22:05.138031+0000","last_clean":"2024-08-17T02:22:05.138031+0000","last_became_active":"2024-08-17T02:22:04.139999+0000","last_became_peered":"2024-08-17T02:22:04.139999+0000","last_unstale":"2024-08-17T02:22:05.138031+0000","last_undegraded":"2024-08-17T02:22:05.138031+0000","last_fullsized":"2024-08-17T02:22:05.138031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:20:55.997912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136517+0000","last_change":"2024-08-17T02:22:05.129259+0000","last_active":"2024-08-17T02:22:05.136517+0000","last_peered":"2024-08-17T02:22:05.136517+0000","last_clean":"2024-08-17T02:22:05.136517+0000","last_became_active":"2024-08-17T02:22:05.128993+0000","last_became_peered":"2024-08-17T02:22:05.128993+0000","last_unstale":"2024-08-17T02:22:05.136517+0000","last_undegraded":"2024-08-17T02:22:05.136517+0000","last_fullsized":"2024-08-17T02:22:05.136517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:01:17.239249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.120637+0000","last_change":"2024-08-17T02:22:04.136465+0000","last_active":"2024-08-17T02:22:05.120637+0000","last_peered":"2024-08-17T02:22:05.120637+0000","last_clean":"2024-08-17T02:22:05.120637+0000","last_became_active":"2024-08-17T02:22:04.136153+0000","last_became_peered":"2024-08-17T02:22:04.136153+0000","last_unstale":"2024-08-17T02:22:05.120637+0000","last_undegraded":"2024-08-17T02:22:05.120637+0000","last_fullsized":"2024-08-17T02:22:05.120637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:25:44.685521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.130033+0000","last_change":"2024-08-17T02:22:05.125665+0000","last_active":"2024-08-17T02:22:05.130033+0000","last_peered":"2024-08-17T02:22:05.130033+0000","last_clean":"2024-08-17T02:22:05.130033+0000","last_became_active":"2024-08-17T02:22:05.125115+0000","last_became_peered":"2024-08-17T02:22:05.125115+0000","last_unstale":"2024-08-17T02:22:05.130033+0000","last_undegraded":"2024-08-17T02:22:05.130033+0000","last_fullsized":"2024-08-17T02:22:05.130033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:57.328385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110749+0000","last_change":"2024-08-17T02:22:04.125704+0000","last_active":"2024-08-17T02:22:05.110749+0000","last_peered":"2024-08-17T02:22:05.110749+0000","last_clean":"2024-08-17T02:22:05.110749+0000","last_became_active":"2024-08-17T02:22:04.124384+0000","last_became_peered":"2024-08-17T02:22:04.124384+0000","last_unstale":"2024-08-17T02:22:05.110749+0000","last_undegraded":"2024-08-17T02:22:05.110749+0000","last_fullsized":"2024-08-17T02:22:05.110749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:51:56.324601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144755+0000","last_change":"2024-08-17T02:22:05.140080+0000","last_active":"2024-08-17T02:22:05.144755+0000","last_peered":"2024-08-17T02:22:05.144755+0000","last_clean":"2024-08-17T02:22:05.144755+0000","last_became_active":"2024-08-17T02:22:05.139642+0000","last_became_peered":"2024-08-17T02:22:05.139642+0000","last_unstale":"2024-08-17T02:22:05.144755+0000","last_undegraded":"2024-08-17T02:22:05.144755+0000","last_fullsized":"2024-08-17T02:22:05.144755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:33:56.139125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.110708+0000","last_change":"2024-08-17T02:22:04.128418+0000","last_active":"2024-08-17T02:22:05.110708+0000","last_peered":"2024-08-17T02:22:05.110708+0000","last_clean":"2024-08-17T02:22:05.110708+0000","last_became_active":"2024-08-17T02:22:04.128178+0000","last_became_peered":"2024-08-17T02:22:04.128178+0000","last_unstale":"2024-08-17T02:22:05.110708+0000","last_undegraded":"2024-08-17T02:22:05.110708+0000","last_fullsized":"2024-08-17T02:22:05.110708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:46:45.638229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.135568+0000","last_change":"2024-08-17T02:22:05.132678+0000","last_active":"2024-08-17T02:22:05.135568+0000","last_peered":"2024-08-17T02:22:05.135568+0000","last_clean":"2024-08-17T02:22:05.135568+0000","last_became_active":"2024-08-17T02:22:05.132375+0000","last_became_peered":"2024-08-17T02:22:05.132375+0000","last_unstale":"2024-08-17T02:22:05.135568+0000","last_undegraded":"2024-08-17T02:22:05.135568+0000","last_fullsized":"2024-08-17T02:22:05.135568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:05:58.104311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144474+0000","last_change":"2024-08-17T02:22:04.120559+0000","last_active":"2024-08-17T02:22:05.144474+0000","last_peered":"2024-08-17T02:22:05.144474+0000","last_clean":"2024-08-17T02:22:05.144474+0000","last_became_active":"2024-08-17T02:22:04.120229+0000","last_became_peered":"2024-08-17T02:22:04.120229+0000","last_unstale":"2024-08-17T02:22:05.144474+0000","last_undegraded":"2024-08-17T02:22:05.144474+0000","last_fullsized":"2024-08-17T02:22:05.144474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:40.464171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.133762+0000","last_change":"2024-08-17T02:22:05.129982+0000","last_active":"2024-08-17T02:22:05.133762+0000","last_peered":"2024-08-17T02:22:05.133762+0000","last_clean":"2024-08-17T02:22:05.133762+0000","last_became_active":"2024-08-17T02:22:05.129645+0000","last_became_peered":"2024-08-17T02:22:05.129645+0000","last_unstale":"2024-08-17T02:22:05.133762+0000","last_undegraded":"2024-08-17T02:22:05.133762+0000","last_fullsized":"2024-08-17T02:22:05.133762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:32:11.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.141475+0000","last_change":"2024-08-17T02:22:05.130378+0000","last_active":"2024-08-17T02:22:05.141475+0000","last_peered":"2024-08-17T02:22:05.141475+0000","last_clean":"2024-08-17T02:22:05.141475+0000","last_became_active":"2024-08-17T02:22:05.130027+0000","last_became_peered":"2024-08-17T02:22:05.130027+0000","last_unstale":"2024-08-17T02:22:05.141475+0000","last_undegraded":"2024-08-17T02:22:05.141475+0000","last_fullsized":"2024-08-17T02:22:05.141475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:44:40.942386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.136607+0000","last_change":"2024-08-17T02:22:04.126856+0000","last_active":"2024-08-17T02:22:05.136607+0000","last_peered":"2024-08-17T02:22:05.136607+0000","last_clean":"2024-08-17T02:22:05.136607+0000","last_became_active":"2024-08-17T02:22:04.126479+0000","last_became_peered":"2024-08-17T02:22:04.126479+0000","last_unstale":"2024-08-17T02:22:05.136607+0000","last_undegraded":"2024-08-17T02:22:05.136607+0000","last_fullsized":"2024-08-17T02:22:05.136607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:37:49.976157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139018+0000","last_change":"2024-08-17T02:22:05.129249+0000","last_active":"2024-08-17T02:22:05.139018+0000","last_peered":"2024-08-17T02:22:05.139018+0000","last_clean":"2024-08-17T02:22:05.139018+0000","last_became_active":"2024-08-17T02:22:05.127468+0000","last_became_peered":"2024-08-17T02:22:05.127468+0000","last_unstale":"2024-08-17T02:22:05.139018+0000","last_undegraded":"2024-08-17T02:22:05.139018+0000","last_fullsized":"2024-08-17T02:22:05.139018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:48:12.922294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.137386+0000","last_change":"2024-08-17T02:22:04.130183+0000","last_active":"2024-08-17T02:22:05.137386+0000","last_peered":"2024-08-17T02:22:05.137386+0000","last_clean":"2024-08-17T02:22:05.137386+0000","last_became_active":"2024-08-17T02:22:04.129922+0000","last_became_peered":"2024-08-17T02:22:04.129922+0000","last_unstale":"2024-08-17T02:22:05.137386+0000","last_undegraded":"2024-08-17T02:22:05.137386+0000","last_fullsized":"2024-08-17T02:22:05.137386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:36:14.299671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.139349+0000","last_change":"2024-08-17T02:22:05.131559+0000","last_active":"2024-08-17T02:22:05.139349+0000","last_peered":"2024-08-17T02:22:05.139349+0000","last_clean":"2024-08-17T02:22:05.139349+0000","last_became_active":"2024-08-17T02:22:05.131142+0000","last_became_peered":"2024-08-17T02:22:05.131142+0000","last_unstale":"2024-08-17T02:22:05.139349+0000","last_undegraded":"2024-08-17T02:22:05.139349+0000","last_fullsized":"2024-08-17T02:22:05.139349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:21:52.626562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.134704+0000","last_change":"2024-08-17T02:22:04.136408+0000","last_active":"2024-08-17T02:22:05.134704+0000","last_peered":"2024-08-17T02:22:05.134704+0000","last_clean":"2024-08-17T02:22:05.134704+0000","last_became_active":"2024-08-17T02:22:04.136140+0000","last_became_peered":"2024-08-17T02:22:04.136140+0000","last_unstale":"2024-08-17T02:22:05.134704+0000","last_undegraded":"2024-08-17T02:22:05.134704+0000","last_fullsized":"2024-08-17T02:22:05.134704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:50:52.363435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.138499+0000","last_change":"2024-08-17T02:22:05.134585+0000","last_active":"2024-08-17T02:22:05.138499+0000","last_peered":"2024-08-17T02:22:05.138499+0000","last_clean":"2024-08-17T02:22:05.138499+0000","last_became_active":"2024-08-17T02:22:05.134230+0000","last_became_peered":"2024-08-17T02:22:05.134230+0000","last_unstale":"2024-08-17T02:22:05.138499+0000","last_undegraded":"2024-08-17T02:22:05.138499+0000","last_fullsized":"2024-08-17T02:22:05.138499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:04.089363+0000","last_clean_scrub_stamp":"2024-08-17T02:22:04.089363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:13.731621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:22:05.144051+0000","last_change":"2024-08-17T02:22:04.140683+0000","last_active":"2024-08-17T02:22:05.144051+0000","last_peered":"2024-08-17T02:22:05.144051+0000","last_clean":"2024-08-17T02:22:05.144051+0000","last_became_active":"2024-08-17T02:22:04.140386+0000","last_became_peered":"2024-08-17T02:22:04.140386+0000","last_unstale":"2024-08-17T02:22:05.144051+0000","last_undegraded":"2024-08-17T02:22:05.144051+0000","last_fullsized":"2024-08-17T02:22:05.144051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:03.085579+0000","last_clean_scrub_stamp":"2024-08-17T02:22:03.085579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:07:35.811187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298360,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073480,"statfs":{"total":96636764160,"available":96331243520,"internally_reserved":0,"allocated":823296,"data_stored":669312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":237568,"data_stored":79926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":229376,"data_stored":78464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":237568,"data_stored":78500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.415}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.544}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":237568,"data_stored":79926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":78374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":233472,"data_stored":78534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":831488,"data_stored":669368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:16.817 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-17T02:22:16.818 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-17T02:22:16.818 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-08-17T02:22:16.818 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-08-17T02:22:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 Processor -- start 2024-08-17T02:22:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- start start 2024-08-17T02:22:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcda80b0350 0x7fcda80ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda80af2b0 con 0x7fcda8155d20 2024-08-17T02:22:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda80af590 con 0x7fcda806df10 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda80b07a0 con 0x7fcda80b0350 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda5d9b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcda80b0350 0x7fcda80ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda5d9b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcda80b0350 0x7fcda80ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52534/0 (socket says 172.21.15.100:52534) 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda5d9b700 1 -- 172.21.15.100:0/3264044754 learned_addr learned my addr 172.21.15.100:0/3264044754 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda5d9b700 1 -- 172.21.15.100:0/3264044754 >> v1:172.21.15.123:6789/0 conn(0x7fcda806df10 legacy=0x7fcda806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda5d9b700 1 -- 172.21.15.100:0/3264044754 >> v1:172.21.15.100:6789/0 conn(0x7fcda8155d20 legacy=0x7fcda80aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda5d9b700 1 -- 172.21.15.100:0/3264044754 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda80af870 con 0x7fcda80b0350 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda5d9b700 1 --2- 172.21.15.100:0/3264044754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda80ad7c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd900099b0 tx=0x7fcd90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb534a6dc556a98 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda559a700 1 -- 172.21.15.100:0/3264044754 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000b860 con 0x7fcda80b0350 2024-08-17T02:22:16.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda559a700 1 -- 172.21.15.100:0/3264044754 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9000b9c0 con 0x7fcda80b0350 2024-08-17T02:22:16.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda559a700 1 -- 172.21.15.100:0/3264044754 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000bd60 con 0x7fcda80b0350 2024-08-17T02:22:16.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- 172.21.15.100:0/3264044754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 msgr2=0x7fcda80ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 --2- 172.21.15.100:0/3264044754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda80ad7c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd900099b0 tx=0x7fcd90009250 comp rx=0 tx=0).stop 2024-08-17T02:22:16.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- 172.21.15.100:0/3264044754 shutdown_connections 2024-08-17T02:22:16.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 --2- 172.21.15.100:0/3264044754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda80ad7c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- 172.21.15.100:0/3264044754 >> 172.21.15.100:0/3264044754 conn(0x7fcda80678d0 msgr2=0x7fcda8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:16.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- 172.21.15.100:0/3264044754 shutdown_connections 2024-08-17T02:22:16.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.960+0000 7fcda759e700 1 -- 172.21.15.100:0/3264044754 wait complete. 2024-08-17T02:22:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda759e700 1 Processor -- start 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda759e700 1 -- start start 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda806df10 0x7fcda814f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda8168dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcda8155d20 0x7fcda8157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda759e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda8157560 con 0x7fcda80b0350 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda814fb80 con 0x7fcda806df10 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda8157a50 con 0x7fcda8155d20 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda5d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda8168dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda5d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda8168dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52542/0 (socket says 172.21.15.100:52542) 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.964+0000 7fcda5d9b700 1 -- 172.21.15.100:0/3160121054 learned_addr learned my addr 172.21.15.100:0/3160121054 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda6d9d700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcda8155d20 0x7fcda8157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda659c700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda806df10 0x7fcda814f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda5d9b700 1 -- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcda8155d20 msgr2=0x7fcda8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda5d9b700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcda8155d20 0x7fcda8157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda5d9b700 1 -- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda806df10 msgr2=0x7fcda814f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda5d9b700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda806df10 0x7fcda814f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda5d9b700 1 -- 172.21.15.100:0/3160121054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd900094f0 con 0x7fcda80b0350 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda5d9b700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda8168dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009290 tx=0x7fcd90019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcd9e7fc700 1 -- 172.21.15.100:0/3160121054 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9001e070 con 0x7fcda80b0350 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcd9e7fc700 1 -- 172.21.15.100:0/3160121054 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd90019c50 con 0x7fcda80b0350 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcd9e7fc700 1 -- 172.21.15.100:0/3160121054 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90010470 con 0x7fcda80b0350 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda814b100 con 0x7fcda80b0350 2024-08-17T02:22:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda814b6c0 con 0x7fcda80b0350 2024-08-17T02:22:16.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda8157c10 con 0x7fcda80b0350 2024-08-17T02:22:16.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.968+0000 7fcd9e7fc700 1 -- 172.21.15.100:0/3160121054 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcd90022070 con 0x7fcda80b0350 2024-08-17T02:22:16.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.972+0000 7fcd9e7fc700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcd8c060890 0x7fcd8c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:16.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.972+0000 7fcd9e7fc700 1 -- 172.21.15.100:0/3160121054 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd90014070 con 0x7fcda80b0350 2024-08-17T02:22:16.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.972+0000 7fcda659c700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcd8c060890 0x7fcd8c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:16.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.972+0000 7fcda659c700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcd8c060890 0x7fcd8c062d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd98003a00 tx=0x7fcd98007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:16.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:16.980+0000 7fcd9e7fc700 1 -- 172.21.15.100:0/3160121054 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcda8157c10 con 0x7fcda80b0350 2024-08-17T02:22:17.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.160+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcda8157c10 con 0x7fcda80b0350 2024-08-17T02:22:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.160+0000 7fcd9e7fc700 1 -- 172.21.15.100:0/3160121054 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcda8157c10 con 0x7fcda80b0350 2024-08-17T02:22:17.163 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:17.163 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-17T02:22:17.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcd8c060890 msgr2=0x7fcd8c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcd8c060890 0x7fcd8c062d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd98003a00 tx=0x7fcd98007ec0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 msgr2=0x7fcda8168dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda8168dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009290 tx=0x7fcd90019000 comp rx=0 tx=0).stop 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 shutdown_connections 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcd8c060890 0x7fcd8c062d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcda8155d20 0x7fcda8157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda80b0350 0x7fcda8168dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 --2- 172.21.15.100:0/3160121054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda806df10 0x7fcda814f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 >> 172.21.15.100:0/3160121054 conn(0x7fcda80678d0 msgr2=0x7fcda8069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 shutdown_connections 2024-08-17T02:22:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.164+0000 7fcda759e700 1 -- 172.21.15.100:0/3160121054 wait complete. 2024-08-17T02:22:17.176 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-08-17T02:22:17.176 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:17.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.268+0000 7f5e6669c700 1 Processor -- start 2024-08-17T02:22:17.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.268+0000 7f5e6669c700 1 -- start start 2024-08-17T02:22:17.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e600d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e6015cea0 0x7f5e6015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e600cf190 con 0x7f5e60159040 2024-08-17T02:22:17.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e600d09a0 con 0x7f5e6015cea0 2024-08-17T02:22:17.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600ce5e0 con 0x7f5e600d0010 2024-08-17T02:22:17.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e65e9b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e6015cea0 0x7f5e6015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e65e9b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e6015cea0 0x7f5e6015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52548/0 (socket says 172.21.15.100:52548) 2024-08-17T02:22:17.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e65e9b700 1 -- 172.21.15.100:0/3527108028 learned_addr learned my addr 172.21.15.100:0/3527108028 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e64e99700 1 --2- 172.21.15.100:0/3527108028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e600d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e65e9b700 1 -- 172.21.15.100:0/3527108028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 msgr2=0x7f5e600d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e65e9b700 1 --2- 172.21.15.100:0/3527108028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e600d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e65e9b700 1 -- 172.21.15.100:0/3527108028 >> v1:172.21.15.123:6789/0 conn(0x7f5e60159040 legacy=0x7f5e600cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e65e9b700 1 -- 172.21.15.100:0/3527108028 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600ce860 con 0x7f5e6015cea0 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e65e9b700 1 --2- 172.21.15.100:0/3527108028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6015cea0 0x7f5e6015d320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009f20 tx=0x7f5e58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b00bc3f0eb064e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e57fff700 1 -- 172.21.15.100:0/3527108028 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800e070 con 0x7f5e6015cea0 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e57fff700 1 -- 172.21.15.100:0/3527108028 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5800b9d0 con 0x7f5e6015cea0 2024-08-17T02:22:17.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e57fff700 1 -- 172.21.15.100:0/3527108028 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580092e0 con 0x7f5e6015cea0 2024-08-17T02:22:17.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 -- 172.21.15.100:0/3527108028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6015cea0 msgr2=0x7f5e6015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 --2- 172.21.15.100:0/3527108028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6015cea0 0x7f5e6015d320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009f20 tx=0x7f5e58009840 comp rx=0 tx=0).stop 2024-08-17T02:22:17.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 -- 172.21.15.100:0/3527108028 shutdown_connections 2024-08-17T02:22:17.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 --2- 172.21.15.100:0/3527108028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6015cea0 0x7f5e6015d320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 --2- 172.21.15.100:0/3527108028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e600d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 -- 172.21.15.100:0/3527108028 >> 172.21.15.100:0/3527108028 conn(0x7f5e60068250 msgr2=0x7f5e60068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:17.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 -- 172.21.15.100:0/3527108028 shutdown_connections 2024-08-17T02:22:17.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.272+0000 7f5e6669c700 1 -- 172.21.15.100:0/3527108028 wait complete. 2024-08-17T02:22:17.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.276+0000 7f5e6669c700 1 Processor -- start 2024-08-17T02:22:17.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.276+0000 7f5e6669c700 1 -- start start 2024-08-17T02:22:17.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e601469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e60159040 0x7f5e6015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6015cea0 0x7f5e6015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600b2530 con 0x7f5e60159040 2024-08-17T02:22:17.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600b1920 con 0x7f5e6015cea0 2024-08-17T02:22:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600b1c00 con 0x7f5e600d0010 2024-08-17T02:22:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e60159040 0x7f5e6015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e60159040 0x7f5e6015c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52556/0 (socket says 172.21.15.100:52556) 2024-08-17T02:22:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 -- 172.21.15.100:0/1292720167 learned_addr learned my addr 172.21.15.100:0/1292720167 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6569a700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e601469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e65e9b700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6015cea0 0x7f5e6015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 -- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 msgr2=0x7f5e601469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e601469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 -- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6015cea0 msgr2=0x7f5e6015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6015cea0 0x7f5e6015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 -- 172.21.15.100:0/1292720167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e58009ae0 con 0x7f5e60159040 2024-08-17T02:22:17.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e65e9b700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6015cea0 0x7f5e6015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:17.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6569a700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e601469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:17.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e64e99700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e60159040 0x7f5e6015c0b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000e930 tx=0x7f5e500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e557fa700 1 -- 172.21.15.100:0/1292720167 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000eed0 con 0x7f5e60159040 2024-08-17T02:22:17.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e557fa700 1 -- 172.21.15.100:0/1292720167 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5000c8e0 con 0x7f5e60159040 2024-08-17T02:22:17.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e600b1ee0 con 0x7f5e60159040 2024-08-17T02:22:17.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e557fa700 1 -- 172.21.15.100:0/1292720167 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000cc80 con 0x7f5e60159040 2024-08-17T02:22:17.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e60158530 con 0x7f5e60159040 2024-08-17T02:22:17.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.280+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e600b26e0 con 0x7f5e60159040 2024-08-17T02:22:17.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.284+0000 7f5e557fa700 1 -- 172.21.15.100:0/1292720167 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5e50013680 con 0x7f5e60159040 2024-08-17T02:22:17.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.284+0000 7f5e557fa700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e44060890 0x7f5e44062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.284+0000 7f5e557fa700 1 -- 172.21.15.100:0/1292720167 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e5007cdb0 con 0x7f5e60159040 2024-08-17T02:22:17.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.284+0000 7f5e6569a700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e44060890 0x7f5e44062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.284+0000 7f5e6569a700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e44060890 0x7f5e44062d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c007fd0 tx=0x7f5e5c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.292+0000 7f5e557fa700 1 -- 172.21.15.100:0/1292720167 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5e600b26e0 con 0x7f5e60159040 2024-08-17T02:22:17.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.448+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e600643a0 con 0x7f5e60159040 2024-08-17T02:22:17.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.448+0000 7f5e557fa700 1 -- 172.21.15.100:0/1292720167 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f5e50050c90 con 0x7f5e60159040 2024-08-17T02:22:17.451 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:17.451 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:17.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e44060890 msgr2=0x7f5e44062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e44060890 0x7f5e44062d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c007fd0 tx=0x7f5e5c009040 comp rx=0 tx=0).stop 2024-08-17T02:22:17.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e60159040 msgr2=0x7f5e6015c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e60159040 0x7f5e6015c0b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000e930 tx=0x7f5e500091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 shutdown_connections 2024-08-17T02:22:17.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e44060890 0x7f5e44062d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6015cea0 0x7f5e6015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e60159040 0x7f5e6015c0b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 --2- 172.21.15.100:0/1292720167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e600d0010 0x7f5e601469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 >> 172.21.15.100:0/1292720167 conn(0x7f5e60068250 msgr2=0x7f5e60156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:17.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 shutdown_connections 2024-08-17T02:22:17.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.452+0000 7f5e6669c700 1 -- 172.21.15.100:0/1292720167 wait complete. 2024-08-17T02:22:17.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:02.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:03.458625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:22:17.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.568+0000 7faa9dcef700 1 Processor -- start 2024-08-17T02:22:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.568+0000 7faa9dcef700 1 -- start start 2024-08-17T02:22:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa9815a680 0x7faa9806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa981570b0 0x7faa9806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa9815a1f0 con 0x7faa98153a30 2024-08-17T02:22:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa980af0c0 con 0x7faa981570b0 2024-08-17T02:22:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa98153e00 con 0x7faa9815a680 2024-08-17T02:22:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9d4ee700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa981570b0 0x7faa9806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9d4ee700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa981570b0 0x7faa9806c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52560/0 (socket says 172.21.15.100:52560) 2024-08-17T02:22:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9d4ee700 1 -- 172.21.15.100:0/3567108987 learned_addr learned my addr 172.21.15.100:0/3567108987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa97fff700 1 --2- 172.21.15.100:0/3567108987 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa9815a680 0x7faa9806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9d4ee700 1 -- 172.21.15.100:0/3567108987 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa9815a680 msgr2=0x7faa9806c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9d4ee700 1 --2- 172.21.15.100:0/3567108987 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa9815a680 0x7faa9806c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9d4ee700 1 -- 172.21.15.100:0/3567108987 >> v1:172.21.15.123:6789/0 conn(0x7faa98153a30 legacy=0x7faa980ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9d4ee700 1 -- 172.21.15.100:0/3567108987 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980aede0 con 0x7faa981570b0 2024-08-17T02:22:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9d4ee700 1 --2- 172.21.15.100:0/3567108987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa981570b0 0x7faa9806c740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa90009f20 tx=0x7faa90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6011bce6dba713 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa977fe700 1 -- 172.21.15.100:0/3567108987 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9000e070 con 0x7faa981570b0 2024-08-17T02:22:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa977fe700 1 -- 172.21.15.100:0/3567108987 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9000b9d0 con 0x7faa981570b0 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa977fe700 1 -- 172.21.15.100:0/3567108987 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa900092e0 con 0x7faa981570b0 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 -- 172.21.15.100:0/3567108987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa981570b0 msgr2=0x7faa9806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 --2- 172.21.15.100:0/3567108987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa981570b0 0x7faa9806c740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa90009f20 tx=0x7faa90009840 comp rx=0 tx=0).stop 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 -- 172.21.15.100:0/3567108987 shutdown_connections 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 --2- 172.21.15.100:0/3567108987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa981570b0 0x7faa9806c740 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 --2- 172.21.15.100:0/3567108987 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa9815a680 0x7faa9806c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 -- 172.21.15.100:0/3567108987 >> 172.21.15.100:0/3567108987 conn(0x7faa980ca2b0 msgr2=0x7faa9806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 -- 172.21.15.100:0/3567108987 shutdown_connections 2024-08-17T02:22:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.572+0000 7faa9dcef700 1 -- 172.21.15.100:0/3567108987 wait complete. 2024-08-17T02:22:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9dcef700 1 Processor -- start 2024-08-17T02:22:17.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9dcef700 1 -- start start 2024-08-17T02:22:17.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9dcef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa98153a30 0x7faa98139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9dcef700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa981570b0 0x7faa981462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9dcef700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9815a680 0x7faa98146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9dcef700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980cc1a0 con 0x7faa9815a680 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9dcef700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980cc480 con 0x7faa98153a30 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9dcef700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980cc760 con 0x7faa981570b0 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9d4ee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9815a680 0x7faa98146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9d4ee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9815a680 0x7faa98146810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52568/0 (socket says 172.21.15.100:52568) 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.576+0000 7faa9d4ee700 1 -- 172.21.15.100:0/2919868503 learned_addr learned my addr 172.21.15.100:0/2919868503 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9cced700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa98153a30 0x7faa98139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa97fff700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa981570b0 0x7faa981462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9d4ee700 1 -- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa981570b0 msgr2=0x7faa981462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9d4ee700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa981570b0 0x7faa981462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9d4ee700 1 -- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa98153a30 msgr2=0x7faa98139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9d4ee700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa98153a30 0x7faa98139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9d4ee700 1 -- 172.21.15.100:0/2919868503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa90009ae0 con 0x7faa9815a680 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9d4ee700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9815a680 0x7faa98146810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa9806dde0 tx=0x7faa90018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa94ff9700 1 -- 172.21.15.100:0/2919868503 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90008ad0 con 0x7faa9815a680 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa94ff9700 1 -- 172.21.15.100:0/2919868503 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa90008c30 con 0x7faa9815a680 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa94ff9700 1 -- 172.21.15.100:0/2919868503 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90026460 con 0x7faa9815a680 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9814ba70 con 0x7faa9815a680 2024-08-17T02:22:17.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9814b830 con 0x7faa9815a680 2024-08-17T02:22:17.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa94ff9700 1 -- 172.21.15.100:0/2919868503 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7faa90003040 con 0x7faa9815a680 2024-08-17T02:22:17.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa980638f0 con 0x7faa9815a680 2024-08-17T02:22:17.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa94ff9700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faa78060860 0x7faa78062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa94ff9700 1 -- 172.21.15.100:0/2919868503 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa9007e2e0 con 0x7faa9815a680 2024-08-17T02:22:17.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.580+0000 7faa9cced700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faa78060860 0x7faa78062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.584+0000 7faa9cced700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faa78060860 0x7faa78062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa980b2d00 tx=0x7faa80009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.588+0000 7faa94ff9700 1 -- 172.21.15.100:0/2919868503 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7faa9004e730 con 0x7faa9815a680 2024-08-17T02:22:17.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.760+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa9814b830 con 0x7faa9815a680 2024-08-17T02:22:17.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.760+0000 7faa94ff9700 1 -- 172.21.15.100:0/2919868503 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa9814b830 con 0x7faa9815a680 2024-08-17T02:22:17.764 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:17.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faa78060860 msgr2=0x7faa78062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faa78060860 0x7faa78062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa980b2d00 tx=0x7faa80009040 comp rx=0 tx=0).stop 2024-08-17T02:22:17.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9815a680 msgr2=0x7faa98146810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9815a680 0x7faa98146810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa9806dde0 tx=0x7faa90018000 comp rx=0 tx=0).stop 2024-08-17T02:22:17.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 shutdown_connections 2024-08-17T02:22:17.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faa78060860 0x7faa78062d10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9815a680 0x7faa98146810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa981570b0 0x7faa981462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 --2- 172.21.15.100:0/2919868503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa98153a30 0x7faa98139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 >> 172.21.15.100:0/2919868503 conn(0x7faa980ca2b0 msgr2=0x7faa98154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:17.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 shutdown_connections 2024-08-17T02:22:17.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.764+0000 7faa9dcef700 1 -- 172.21.15.100:0/2919868503 wait complete. 2024-08-17T02:22:17.779 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:22:17.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01c284e700 1 Processor -- start 2024-08-17T02:22:17.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01c284e700 1 -- start start 2024-08-17T02:22:17.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01c284e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc1596d0 0x7f01bc066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01c284e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc0cba90 con 0x7f01bc15dbc0 2024-08-17T02:22:17.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01c284e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc0cb160 con 0x7f01bc159300 2024-08-17T02:22:17.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01c284e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0b2b60 con 0x7f01bc1596d0 2024-08-17T02:22:17.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01c0deb700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f01bc15dbc0 0x7f01bc0cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50948/0 (socket says 172.21.15.100:50948) 2024-08-17T02:22:17.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01c0deb700 1 -- 172.21.15.100:0/292581507 learned_addr learned my addr 172.21.15.100:0/292581507 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:17.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01bb7fe700 1 --2- 172.21.15.100:0/292581507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc1596d0 0x7f01bc066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01baffd700 1 -- 172.21.15.100:0/292581507 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030310194 0 0) 0x7f01bc0cba90 con 0x7f01bc15dbc0 2024-08-17T02:22:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01baffd700 1 -- 172.21.15.100:0/292581507 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01a4003570 con 0x7f01bc15dbc0 2024-08-17T02:22:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01bb7fe700 1 -- 172.21.15.100:0/292581507 >> v1:172.21.15.123:6789/0 conn(0x7f01bc159300 legacy=0x7f01bc0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01bb7fe700 1 -- 172.21.15.100:0/292581507 >> v1:172.21.15.100:6789/0 conn(0x7f01bc15dbc0 legacy=0x7f01bc0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.888+0000 7f01bb7fe700 1 -- 172.21.15.100:0/292581507 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc0cb440 con 0x7f01bc1596d0 2024-08-17T02:22:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01bb7fe700 1 --2- 172.21.15.100:0/292581507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc1596d0 0x7f01bc066e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009910 tx=0x7f01ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca4147885e7ce6e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01baffd700 1 -- 172.21.15.100:0/292581507 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00b680 con 0x7f01bc1596d0 2024-08-17T02:22:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01baffd700 1 -- 172.21.15.100:0/292581507 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00b7e0 con 0x7f01bc1596d0 2024-08-17T02:22:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01baffd700 1 -- 172.21.15.100:0/292581507 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00bb60 con 0x7f01bc1596d0 2024-08-17T02:22:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- 172.21.15.100:0/292581507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc1596d0 msgr2=0x7f01bc066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 --2- 172.21.15.100:0/292581507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc1596d0 0x7f01bc066e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009910 tx=0x7f01ac0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- 172.21.15.100:0/292581507 shutdown_connections 2024-08-17T02:22:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 --2- 172.21.15.100:0/292581507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc1596d0 0x7f01bc066e00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- 172.21.15.100:0/292581507 >> 172.21.15.100:0/292581507 conn(0x7f01bc0682c0 msgr2=0x7f01bc0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:17.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- 172.21.15.100:0/292581507 shutdown_connections 2024-08-17T02:22:17.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- 172.21.15.100:0/292581507 wait complete. 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 Processor -- start 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- start start 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc159300 0x7f01bc1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01bc1596d0 0x7f01bc15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc15dbc0 0x7f01bc0d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc155a20 con 0x7f01bc1596d0 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc15bc50 con 0x7f01bc159300 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c284e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0c8e70 con 0x7f01bc15dbc0 2024-08-17T02:22:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01bb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01bc1596d0 0x7f01bc15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01bb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01bc1596d0 0x7f01bc15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52576/0 (socket says 172.21.15.100:52576) 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01bb7fe700 1 -- 172.21.15.100:0/2859184295 learned_addr learned my addr 172.21.15.100:0/2859184295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01c0deb700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc15dbc0 0x7f01bc0d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.892+0000 7f01bbfff700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc159300 0x7f01bc1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01bb7fe700 1 -- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc15dbc0 msgr2=0x7f01bc0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01bb7fe700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc15dbc0 0x7f01bc0d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01bb7fe700 1 -- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc159300 msgr2=0x7f01bc1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01bb7fe700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc159300 0x7f01bc1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01bb7fe700 1 -- 172.21.15.100:0/2859184295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac009420 con 0x7f01bc1596d0 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01bb7fe700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01bc1596d0 0x7f01bc15a080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0cde20 tx=0x7f01ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01a3fff700 1 -- 172.21.15.100:0/2859184295 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00b950 con 0x7f01bc1596d0 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01a3fff700 1 -- 172.21.15.100:0/2859184295 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00bab0 con 0x7f01bc1596d0 2024-08-17T02:22:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc0c9150 con 0x7f01bc1596d0 2024-08-17T02:22:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01a3fff700 1 -- 172.21.15.100:0/2859184295 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac01a680 con 0x7f01bc1596d0 2024-08-17T02:22:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc0c9710 con 0x7f01bc1596d0 2024-08-17T02:22:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc0645c0 con 0x7f01bc1596d0 2024-08-17T02:22:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01a3fff700 1 -- 172.21.15.100:0/2859184295 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f01ac01aab0 con 0x7f01bc1596d0 2024-08-17T02:22:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01a3fff700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f01a8060860 0x7f01a8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.896+0000 7f01a3fff700 1 -- 172.21.15.100:0/2859184295 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f01ac07efb0 con 0x7f01bc1596d0 2024-08-17T02:22:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.900+0000 7f01bbfff700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f01a8060860 0x7f01a8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.900+0000 7f01bbfff700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f01a8060860 0x7f01a8062d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01b0000c00 tx=0x7f01b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:17.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:17.904+0000 7f01a3fff700 1 -- 172.21.15.100:0/2859184295 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f01ac04f480 con 0x7f01bc1596d0 2024-08-17T02:22:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01bc0caac0 con 0x7f01bc1596d0 2024-08-17T02:22:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01a3fff700 1 -- 172.21.15.100:0/2859184295 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01ac052aa0 con 0x7f01bc1596d0 2024-08-17T02:22:18.070 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:22:18.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f01a8060860 msgr2=0x7f01a8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f01a8060860 0x7f01a8062d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01b0000c00 tx=0x7f01b0009040 comp rx=0 tx=0).stop 2024-08-17T02:22:18.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01bc1596d0 msgr2=0x7f01bc15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01bc1596d0 0x7f01bc15a080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0cde20 tx=0x7f01ac017000 comp rx=0 tx=0).stop 2024-08-17T02:22:18.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 shutdown_connections 2024-08-17T02:22:18.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f01a8060860 0x7f01a8062d10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01bc15dbc0 0x7f01bc0d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01bc1596d0 0x7f01bc15a080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 --2- 172.21.15.100:0/2859184295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc159300 0x7f01bc1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 >> 172.21.15.100:0/2859184295 conn(0x7f01bc0682c0 msgr2=0x7f01bc0b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:18.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 shutdown_connections 2024-08-17T02:22:18.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.068+0000 7f01c284e700 1 -- 172.21.15.100:0/2859184295 wait complete. 2024-08-17T02:22:18.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:22:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a6569700 1 Processor -- start 2024-08-17T02:22:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a6569700 1 -- start start 2024-08-17T02:22:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a6569700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f64a0159300 0x7f64a00b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a6569700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a01596d0 0x7f64a0066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a6569700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a00cba90 con 0x7f64a015dbc0 2024-08-17T02:22:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a6569700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a00b2b60 con 0x7f64a0159300 2024-08-17T02:22:18.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a6569700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00cb160 con 0x7f64a01596d0 2024-08-17T02:22:18.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a4b06700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f64a015dbc0 0x7f64a00afe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50964/0 (socket says 172.21.15.100:50964) 2024-08-17T02:22:18.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f64a4b06700 1 -- 172.21.15.100:0/2724953735 learned_addr learned my addr 172.21.15.100:0/2724953735 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:18.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.200+0000 7f649ffff700 1 --2- 172.21.15.100:0/2724953735 >> v2:172.21.15.100:3300/0 conn(0x7f64a0159300 0x7f64a00b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649f7fe700 1 --2- 172.21.15.100:0/2724953735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a01596d0 0x7f64a0066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649ffff700 1 -- 172.21.15.100:0/2724953735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a01596d0 msgr2=0x7f64a0066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:18.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649ffff700 1 --2- 172.21.15.100:0/2724953735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a01596d0 0x7f64a0066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649ffff700 1 -- 172.21.15.100:0/2724953735 >> v1:172.21.15.100:6789/0 conn(0x7f64a015dbc0 legacy=0x7f64a00afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649effd700 1 -- 172.21.15.100:0/2724953735 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132281299 0 0) 0x7f64a00cba90 con 0x7f64a015dbc0 2024-08-17T02:22:18.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649ffff700 1 -- 172.21.15.100:0/2724953735 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00cb440 con 0x7f64a0159300 2024-08-17T02:22:18.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649ffff700 1 --2- 172.21.15.100:0/2724953735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a00b2230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f64940098e0 tx=0x7f64940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2c414dea367b28 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649effd700 1 -- 172.21.15.100:0/2724953735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649400b640 con 0x7f64a0159300 2024-08-17T02:22:18.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649effd700 1 -- 172.21.15.100:0/2724953735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649400b7a0 con 0x7f64a0159300 2024-08-17T02:22:18.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f649effd700 1 -- 172.21.15.100:0/2724953735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649400bb40 con 0x7f64a0159300 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 -- 172.21.15.100:0/2724953735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 msgr2=0x7f64a00b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 --2- 172.21.15.100:0/2724953735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a00b2230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f64940098e0 tx=0x7f64940091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 -- 172.21.15.100:0/2724953735 shutdown_connections 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 --2- 172.21.15.100:0/2724953735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a01596d0 0x7f64a0066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 --2- 172.21.15.100:0/2724953735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a00b2230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 -- 172.21.15.100:0/2724953735 >> 172.21.15.100:0/2724953735 conn(0x7f64a00682c0 msgr2=0x7f64a00686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 -- 172.21.15.100:0/2724953735 shutdown_connections 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 -- 172.21.15.100:0/2724953735 wait complete. 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 Processor -- start 2024-08-17T02:22:18.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.204+0000 7f64a6569700 1 -- start start 2024-08-17T02:22:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a016c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a01596d0 0x7f64a016a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a015dbc0 0x7f64a01557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a016a900 con 0x7f64a0159300 2024-08-17T02:22:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00b4750 con 0x7f64a01596d0 2024-08-17T02:22:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00b3490 con 0x7f64a015dbc0 2024-08-17T02:22:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a016c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a016c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52604/0 (socket says 172.21.15.100:52604) 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 -- 172.21.15.100:0/2888754147 learned_addr learned my addr 172.21.15.100:0/2888754147 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649f7fe700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a01596d0 0x7f64a016a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a4b06700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a015dbc0 0x7f64a01557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 -- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a015dbc0 msgr2=0x7f64a01557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a015dbc0 0x7f64a01557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 -- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a01596d0 msgr2=0x7f64a016a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a01596d0 0x7f64a016a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 -- 172.21.15.100:0/2888754147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494009420 con 0x7f64a0159300 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a4b06700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a015dbc0 0x7f64a01557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f649ffff700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a016c8e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6494009f80 tx=0x7f6494017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f6487fff700 1 -- 172.21.15.100:0/2888754147 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649400bb20 con 0x7f64a0159300 2024-08-17T02:22:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f6487fff700 1 -- 172.21.15.100:0/2888754147 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6494011420 con 0x7f64a0159300 2024-08-17T02:22:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f6487fff700 1 -- 172.21.15.100:0/2888754147 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494011820 con 0x7f64a0159300 2024-08-17T02:22:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a00b3770 con 0x7f64a0159300 2024-08-17T02:22:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a00b3d30 con 0x7f64a0159300 2024-08-17T02:22:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.208+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a00caac0 con 0x7f64a0159300 2024-08-17T02:22:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.212+0000 7f6487fff700 1 -- 172.21.15.100:0/2888754147 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6494011c50 con 0x7f64a0159300 2024-08-17T02:22:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.212+0000 7f6487fff700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f648c060860 0x7f648c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.212+0000 7f6487fff700 1 -- 172.21.15.100:0/2888754147 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f649407e590 con 0x7f64a0159300 2024-08-17T02:22:18.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.212+0000 7f649f7fe700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f648c060860 0x7f648c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.212+0000 7f649f7fe700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f648c060860 0x7f648c062d10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f64a00ceab0 tx=0x7f6490008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.220+0000 7f6487fff700 1 -- 172.21.15.100:0/2888754147 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f649404e9e0 con 0x7f64a0159300 2024-08-17T02:22:18.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.376+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64a00645a0 con 0x7f64a0159300 2024-08-17T02:22:18.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f6487fff700 1 -- 172.21.15.100:0/2888754147 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6494052000 con 0x7f64a0159300 2024-08-17T02:22:18.381 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:22:18.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f648c060860 msgr2=0x7f648c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f648c060860 0x7f648c062d10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f64a00ceab0 tx=0x7f6490008040 comp rx=0 tx=0).stop 2024-08-17T02:22:18.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 msgr2=0x7f64a016c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a016c8e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6494009f80 tx=0x7f6494017000 comp rx=0 tx=0).stop 2024-08-17T02:22:18.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 shutdown_connections 2024-08-17T02:22:18.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f648c060860 0x7f648c062d10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64a015dbc0 0x7f64a01557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a01596d0 0x7f64a016a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 --2- 172.21.15.100:0/2888754147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a0159300 0x7f64a016c8e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 >> 172.21.15.100:0/2888754147 conn(0x7f64a00682c0 msgr2=0x7f64a0158300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:18.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 shutdown_connections 2024-08-17T02:22:18.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.380+0000 7f64a6569700 1 -- 172.21.15.100:0/2888754147 wait complete. 2024-08-17T02:22:18.395 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:18.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25ed520700 1 Processor -- start 2024-08-17T02:22:18.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25ed520700 1 -- start start 2024-08-17T02:22:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25ed520700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25e815aa30 0x7f25e815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25ed520700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25e80b3b30 0x7f25e80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25ed520700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e815a820 con 0x7f25e80aac50 2024-08-17T02:22:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25ed520700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e815bc90 con 0x7f25e815aa30 2024-08-17T02:22:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25ed520700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e80b4470 con 0x7f25e80b3b30 2024-08-17T02:22:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e6ffd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f25e80aac50 0x7f25e81574a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50974/0 (socket says 172.21.15.100:50974) 2024-08-17T02:22:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e6ffd700 1 -- 172.21.15.100:0/2812342666 learned_addr learned my addr 172.21.15.100:0/2812342666 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e67fc700 1 --2- 172.21.15.100:0/2812342666 >> v2:172.21.15.100:3300/0 conn(0x7f25e815aa30 0x7f25e815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e77fe700 1 --2- 172.21.15.100:0/2812342666 >> v2:172.21.15.123:3300/0 conn(0x7f25e80b3b30 0x7f25e80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e5ffb700 1 -- 172.21.15.100:0/2812342666 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399516648 0 0) 0x7f25d0001610 con 0x7f25e80aac50 2024-08-17T02:22:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e5ffb700 1 -- 172.21.15.100:0/2812342666 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25c8003570 con 0x7f25e80aac50 2024-08-17T02:22:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e67fc700 1 -- 172.21.15.100:0/2812342666 >> v2:172.21.15.123:3300/0 conn(0x7f25e80b3b30 msgr2=0x7f25e80b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:18.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e67fc700 1 --2- 172.21.15.100:0/2812342666 >> v2:172.21.15.123:3300/0 conn(0x7f25e80b3b30 0x7f25e80b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e67fc700 1 -- 172.21.15.100:0/2812342666 >> v1:172.21.15.100:6789/0 conn(0x7f25e80aac50 legacy=0x7f25e81574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e67fc700 1 -- 172.21.15.100:0/2812342666 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e815b010 con 0x7f25e815aa30 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e67fc700 1 --2- 172.21.15.100:0/2812342666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815a2e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25e8159860 tx=0x7f25d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21caff36a5474f37 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e5ffb700 1 -- 172.21.15.100:0/2812342666 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8011630 con 0x7f25e815aa30 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e5ffb700 1 -- 172.21.15.100:0/2812342666 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d80117b0 con 0x7f25e815aa30 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.496+0000 7f25e5ffb700 1 -- 172.21.15.100:0/2812342666 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8011b70 con 0x7f25e815aa30 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- 172.21.15.100:0/2812342666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 msgr2=0x7f25e815a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 --2- 172.21.15.100:0/2812342666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815a2e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25e8159860 tx=0x7f25d80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- 172.21.15.100:0/2812342666 shutdown_connections 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 --2- 172.21.15.100:0/2812342666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815a2e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 --2- 172.21.15.100:0/2812342666 >> v2:172.21.15.123:3300/0 conn(0x7f25e80b3b30 0x7f25e80b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- 172.21.15.100:0/2812342666 >> 172.21.15.100:0/2812342666 conn(0x7f25e80994d0 msgr2=0x7f25e80675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:18.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- 172.21.15.100:0/2812342666 shutdown_connections 2024-08-17T02:22:18.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- 172.21.15.100:0/2812342666 wait complete. 2024-08-17T02:22:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 Processor -- start 2024-08-17T02:22:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- start start 2024-08-17T02:22:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e80aac50 0x7f25e81584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25e80b3b30 0x7f25e80abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80ab670 con 0x7f25e815aa30 2024-08-17T02:22:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e809ba90 con 0x7f25e80aac50 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25ed520700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e809bd70 con 0x7f25e80b3b30 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52636/0 (socket says 172.21.15.100:52636) 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25e77fe700 1 -- 172.21.15.100:0/2048935999 learned_addr learned my addr 172.21.15.100:0/2048935999 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25e6ffd700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e80aac50 0x7f25e81584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.500+0000 7f25e67fc700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25e80b3b30 0x7f25e80abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25e77fe700 1 -- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25e80b3b30 msgr2=0x7f25e80abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25e77fe700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25e80b3b30 0x7f25e80abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25e77fe700 1 -- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e80aac50 msgr2=0x7f25e81584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25e77fe700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e80aac50 0x7f25e81584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25e77fe700 1 -- 172.21.15.100:0/2048935999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8009420 con 0x7f25e815aa30 2024-08-17T02:22:18.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25e77fe700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815f240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25e000ea50 tx=0x7f25e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25deffd700 1 -- 172.21.15.100:0/2048935999 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000ee90 con 0x7f25e815aa30 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25deffd700 1 -- 172.21.15.100:0/2048935999 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e000c850 con 0x7f25e815aa30 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e80ab020 con 0x7f25e815aa30 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25deffd700 1 -- 172.21.15.100:0/2048935999 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000cc10 con 0x7f25e815aa30 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81589f0 con 0x7f25e815aa30 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e80ab7e0 con 0x7f25e815aa30 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25deffd700 1 -- 172.21.15.100:0/2048935999 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f25e0013470 con 0x7f25e815aa30 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25deffd700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f25cc060860 0x7f25cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.504+0000 7f25deffd700 1 -- 172.21.15.100:0/2048935999 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f25e007ce00 con 0x7f25e815aa30 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.508+0000 7f25e6ffd700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f25cc060860 0x7f25cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.508+0000 7f25e6ffd700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f25cc060860 0x7f25cc062d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f25d0001770 tx=0x7f25d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.516+0000 7f25deffd700 1 -- 172.21.15.100:0/2048935999 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f25e004d250 con 0x7f25e815aa30 2024-08-17T02:22:18.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.712+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25e80645c0 con 0x7f25e815aa30 2024-08-17T02:22:18.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.712+0000 7f25deffd700 1 -- 172.21.15.100:0/2048935999 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f25e0050870 con 0x7f25e815aa30 2024-08-17T02:22:18.720 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:18.720 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:04.459540+0000","modified":"2024-08-17T02:22:07.136998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:22:18.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f25cc060860 msgr2=0x7f25cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f25cc060860 0x7f25cc062d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f25d0001770 tx=0x7f25d0009040 comp rx=0 tx=0).stop 2024-08-17T02:22:18.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 msgr2=0x7f25e815f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815f240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25e000ea50 tx=0x7f25e00091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 shutdown_connections 2024-08-17T02:22:18.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f25cc060860 0x7f25cc062d10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e815aa30 0x7f25e815f240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25e80b3b30 0x7f25e80abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 --2- 172.21.15.100:0/2048935999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e80aac50 0x7f25e81584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 >> 172.21.15.100:0/2048935999 conn(0x7f25e80994d0 msgr2=0x7f25e80b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:18.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 shutdown_connections 2024-08-17T02:22:18.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.716+0000 7f25ed520700 1 -- 172.21.15.100:0/2048935999 wait complete. 2024-08-17T02:22:18.722 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-08-17T02:22:18.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:18.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.828+0000 7f8761712700 1 Processor -- start 2024-08-17T02:22:18.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.828+0000 7f8761712700 1 -- start start 2024-08-17T02:22:18.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.828+0000 7f8761712700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f875c0b0d20 0x7f875c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.828+0000 7f8761712700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b1800 0x7f875c141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.828+0000 7f8761712700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875c141a00 con 0x7f875c06e450 2024-08-17T02:22:18.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.828+0000 7f8761712700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875c06e8a0 con 0x7f875c0b0d20 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c140b10 con 0x7f875c0b1800 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f875c0b0d20 0x7f875c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f875c0b0d20 0x7f875c06df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52652/0 (socket says 172.21.15.100:52652) 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875bfff700 1 -- 172.21.15.100:0/2902515515 learned_addr learned my addr 172.21.15.100:0/2902515515 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8760f11700 1 --2- 172.21.15.100:0/2902515515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b1800 0x7f875c141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875bfff700 1 -- 172.21.15.100:0/2902515515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b1800 msgr2=0x7f875c141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875bfff700 1 --2- 172.21.15.100:0/2902515515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b1800 0x7f875c141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875bfff700 1 -- 172.21.15.100:0/2902515515 >> v1:172.21.15.123:6789/0 conn(0x7f875c06e450 legacy=0x7f875c0b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875bfff700 1 -- 172.21.15.100:0/2902515515 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c140df0 con 0x7f875c0b0d20 2024-08-17T02:22:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875bfff700 1 --2- 172.21.15.100:0/2902515515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c0b0d20 0x7f875c06df10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87540099b0 tx=0x7f8754009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca3fe2812784b5e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875affd700 1 -- 172.21.15.100:0/2902515515 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754009ea0 con 0x7f875c0b0d20 2024-08-17T02:22:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875affd700 1 -- 172.21.15.100:0/2902515515 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875400b860 con 0x7f875c0b0d20 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f875affd700 1 -- 172.21.15.100:0/2902515515 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400bc00 con 0x7f875c0b0d20 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 -- 172.21.15.100:0/2902515515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c0b0d20 msgr2=0x7f875c06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 --2- 172.21.15.100:0/2902515515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c0b0d20 0x7f875c06df10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87540099b0 tx=0x7f8754009250 comp rx=0 tx=0).stop 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 -- 172.21.15.100:0/2902515515 shutdown_connections 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 --2- 172.21.15.100:0/2902515515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b1800 0x7f875c141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 --2- 172.21.15.100:0/2902515515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c0b0d20 0x7f875c06df10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 -- 172.21.15.100:0/2902515515 >> 172.21.15.100:0/2902515515 conn(0x7f875c0678d0 msgr2=0x7f875c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 -- 172.21.15.100:0/2902515515 shutdown_connections 2024-08-17T02:22:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.832+0000 7f8761712700 1 -- 172.21.15.100:0/2902515515 wait complete. 2024-08-17T02:22:18.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 Processor -- start 2024-08-17T02:22:18.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 -- start start 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c06e450 0x7f875c142e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b0d20 0x7f875c1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875c0b1800 0x7f875c143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c145980 con 0x7f875c06e450 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c144cb0 con 0x7f875c0b1800 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c144fe0 con 0x7f875c0b0d20 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c06e450 0x7f875c142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c06e450 0x7f875c142e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52660/0 (socket says 172.21.15.100:52660) 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 -- 172.21.15.100:0/1032941906 learned_addr learned my addr 172.21.15.100:0/1032941906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8760f11700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875c0b1800 0x7f875c143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875b7fe700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b0d20 0x7f875c1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 -- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b0d20 msgr2=0x7f875c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b0d20 0x7f875c1433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 -- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875c0b1800 msgr2=0x7f875c143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:18.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875c0b1800 0x7f875c143f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:18.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 -- 172.21.15.100:0/1032941906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540094f0 con 0x7f875c06e450 2024-08-17T02:22:18.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f875bfff700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c06e450 0x7f875c142e60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f875c06c2d0 tx=0x7f8754017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8753fff700 1 -- 172.21.15.100:0/1032941906 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754000d10 con 0x7f875c06e450 2024-08-17T02:22:18.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8753fff700 1 -- 172.21.15.100:0/1032941906 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875400bbe0 con 0x7f875c06e450 2024-08-17T02:22:18.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8753fff700 1 -- 172.21.15.100:0/1032941906 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754011670 con 0x7f875c06e450 2024-08-17T02:22:18.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.836+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875c1452c0 con 0x7f875c06e450 2024-08-17T02:22:18.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.840+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875c144480 con 0x7f875c06e450 2024-08-17T02:22:18.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.840+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875c0643a0 con 0x7f875c06e450 2024-08-17T02:22:18.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.840+0000 7f8753fff700 1 -- 172.21.15.100:0/1032941906 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8754011aa0 con 0x7f875c06e450 2024-08-17T02:22:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.840+0000 7f8753fff700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f873c060860 0x7f873c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.840+0000 7f8753fff700 1 -- 172.21.15.100:0/1032941906 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f875407e020 con 0x7f875c06e450 2024-08-17T02:22:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.840+0000 7f875b7fe700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f873c060860 0x7f873c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.844+0000 7f875b7fe700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f873c060860 0x7f873c062d10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f875c06b140 tx=0x7f8744007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:18.848+0000 7f8753fff700 1 -- 172.21.15.100:0/1032941906 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f875404e4f0 con 0x7f875c06e450 2024-08-17T02:22:19.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.012+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f875c1408f0 con 0x7f875c06e450 2024-08-17T02:22:19.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8753fff700 1 -- 172.21.15.100:0/1032941906 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f8754051b10 con 0x7f875c06e450 2024-08-17T02:22:19.017 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:19.017 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:19.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f873c060860 msgr2=0x7f873c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f873c060860 0x7f873c062d10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f875c06b140 tx=0x7f8744007ec0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c06e450 msgr2=0x7f875c142e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c06e450 0x7f875c142e60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f875c06c2d0 tx=0x7f8754017000 comp rx=0 tx=0).stop 2024-08-17T02:22:19.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 shutdown_connections 2024-08-17T02:22:19.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f873c060860 0x7f873c062d10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875c0b1800 0x7f875c143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f875c0b0d20 0x7f875c1433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 --2- 172.21.15.100:0/1032941906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c06e450 0x7f875c142e60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 >> 172.21.15.100:0/1032941906 conn(0x7f875c0678d0 msgr2=0x7f875c15eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:19.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 shutdown_connections 2024-08-17T02:22:19.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.016+0000 7f8761712700 1 -- 172.21.15.100:0/1032941906 wait complete. 2024-08-17T02:22:19.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:02.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:03.458625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:22:19.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:19.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe41826700 1 Processor -- start 2024-08-17T02:22:19.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe41826700 1 -- start start 2024-08-17T02:22:19.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe41826700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe3c156e90 0x7fbe3c0cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe41826700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbe3c157260 0x7fbe3c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe41826700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 0x7fbe3c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe41826700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c0cc580 con 0x7fbe3c156e90 2024-08-17T02:22:19.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe41826700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c0cc860 con 0x7fbe3c157260 2024-08-17T02:22:19.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe41826700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0ccb40 con 0x7fbe3c0aeb40 2024-08-17T02:22:19.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe3affd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe3c156e90 0x7fbe3c0cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe3affd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe3c156e90 0x7fbe3c0cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52672/0 (socket says 172.21.15.100:52672) 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe3affd700 1 -- 172.21.15.100:0/3458694247 learned_addr learned my addr 172.21.15.100:0/3458694247 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.144+0000 7fbe3b7fe700 1 --2- 172.21.15.100:0/3458694247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 0x7fbe3c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe3a7fc700 1 --2- 172.21.15.100:0/3458694247 >> v2:172.21.15.123:3300/0 conn(0x7fbe3c157260 0x7fbe3c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe3affd700 1 -- 172.21.15.100:0/3458694247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 msgr2=0x7fbe3c06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe3affd700 1 --2- 172.21.15.100:0/3458694247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 0x7fbe3c06d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe3affd700 1 -- 172.21.15.100:0/3458694247 >> v2:172.21.15.123:3300/0 conn(0x7fbe3c157260 msgr2=0x7fbe3c06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe3affd700 1 --2- 172.21.15.100:0/3458694247 >> v2:172.21.15.123:3300/0 conn(0x7fbe3c157260 0x7fbe3c06d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe3affd700 1 -- 172.21.15.100:0/3458694247 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0cce20 con 0x7fbe3c156e90 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe3affd700 1 --2- 172.21.15.100:0/3458694247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c156e90 0x7fbe3c0cbcc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c15ab90 tx=0x7fbe34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad8fd45ac572726 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe39ffb700 1 -- 172.21.15.100:0/3458694247 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400e070 con 0x7fbe3c156e90 2024-08-17T02:22:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe39ffb700 1 -- 172.21.15.100:0/3458694247 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3400b970 con 0x7fbe3c156e90 2024-08-17T02:22:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe39ffb700 1 -- 172.21.15.100:0/3458694247 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340092e0 con 0x7fbe3c156e90 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 -- 172.21.15.100:0/3458694247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c156e90 msgr2=0x7fbe3c0cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 --2- 172.21.15.100:0/3458694247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c156e90 0x7fbe3c0cbcc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c15ab90 tx=0x7fbe34009840 comp rx=0 tx=0).stop 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 -- 172.21.15.100:0/3458694247 shutdown_connections 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 --2- 172.21.15.100:0/3458694247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 0x7fbe3c06d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 --2- 172.21.15.100:0/3458694247 >> v2:172.21.15.123:3300/0 conn(0x7fbe3c157260 0x7fbe3c06d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 --2- 172.21.15.100:0/3458694247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c156e90 0x7fbe3c0cbcc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 -- 172.21.15.100:0/3458694247 >> 172.21.15.100:0/3458694247 conn(0x7fbe3c0ceb30 msgr2=0x7fbe3c06cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 -- 172.21.15.100:0/3458694247 shutdown_connections 2024-08-17T02:22:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 -- 172.21.15.100:0/3458694247 wait complete. 2024-08-17T02:22:19.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 Processor -- start 2024-08-17T02:22:19.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.148+0000 7fbe41826700 1 -- start start 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe41826700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 0x7fbe3c13d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe41826700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe3c156e90 0x7fbe3c1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe41826700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c157260 0x7fbe3c168570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe41826700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c168ab0 con 0x7fbe3c157260 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe41826700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c150fb0 con 0x7fbe3c156e90 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe41826700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c169150 con 0x7fbe3c0aeb40 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c157260 0x7fbe3c168570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3affd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 0x7fbe3c13d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c157260 0x7fbe3c168570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52674/0 (socket says 172.21.15.100:52674) 2024-08-17T02:22:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 -- 172.21.15.100:0/1217374750 learned_addr learned my addr 172.21.15.100:0/1217374750 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3a7fc700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe3c156e90 0x7fbe3c1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 -- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 msgr2=0x7fbe3c13d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 0x7fbe3c13d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 -- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe3c156e90 msgr2=0x7fbe3c1536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe3c156e90 0x7fbe3c1536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 -- 172.21.15.100:0/1217374750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe34009ae0 con 0x7fbe3c157260 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe3b7fe700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c157260 0x7fbe3c168570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00ea50 tx=0x7fbe2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe32ffd700 1 -- 172.21.15.100:0/1217374750 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00ee90 con 0x7fbe3c157260 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe32ffd700 1 -- 172.21.15.100:0/1217374750 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00c850 con 0x7fbe3c157260 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c169430 con 0x7fbe3c157260 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe32ffd700 1 -- 172.21.15.100:0/1217374750 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00cc10 con 0x7fbe3c157260 2024-08-17T02:22:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.152+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c1699f0 con 0x7fbe3c157260 2024-08-17T02:22:19.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.156+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c0645c0 con 0x7fbe3c157260 2024-08-17T02:22:19.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.156+0000 7fbe32ffd700 1 -- 172.21.15.100:0/1217374750 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbe2c013470 con 0x7fbe3c157260 2024-08-17T02:22:19.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.156+0000 7fbe32ffd700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe1c060860 0x7fbe1c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.156+0000 7fbe32ffd700 1 -- 172.21.15.100:0/1217374750 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbe2c07cef0 con 0x7fbe3c157260 2024-08-17T02:22:19.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.156+0000 7fbe3affd700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe1c060860 0x7fbe1c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.156+0000 7fbe3affd700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe1c060860 0x7fbe1c062d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009880 tx=0x7fbe34018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.164+0000 7fbe32ffd700 1 -- 172.21.15.100:0/1217374750 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbe2c04d2a0 con 0x7fbe3c157260 2024-08-17T02:22:19.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe3c06f9f0 con 0x7fbe3c157260 2024-08-17T02:22:19.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe32ffd700 1 -- 172.21.15.100:0/1217374750 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe2c0508c0 con 0x7fbe3c157260 2024-08-17T02:22:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:19.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe1c060860 msgr2=0x7fbe1c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe1c060860 0x7fbe1c062d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009880 tx=0x7fbe34018000 comp rx=0 tx=0).stop 2024-08-17T02:22:19.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c157260 msgr2=0x7fbe3c168570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c157260 0x7fbe3c168570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00ea50 tx=0x7fbe2c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 shutdown_connections 2024-08-17T02:22:19.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe1c060860 0x7fbe1c062d10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c157260 0x7fbe3c168570 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe3c156e90 0x7fbe3c1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 --2- 172.21.15.100:0/1217374750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe3c0aeb40 0x7fbe3c13d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 >> 172.21.15.100:0/1217374750 conn(0x7fbe3c0ceb30 msgr2=0x7fbe3c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:19.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 shutdown_connections 2024-08-17T02:22:19.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.328+0000 7fbe41826700 1 -- 172.21.15.100:0/1217374750 wait complete. 2024-08-17T02:22:19.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:22:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a7644c700 1 Processor -- start 2024-08-17T02:22:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a7644c700 1 -- start start 2024-08-17T02:22:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a7644c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a700cfb00 0x7f8a70067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a7644c700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a700cf670 con 0x7f8a70158cc0 2024-08-17T02:22:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a7644c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a700cea60 con 0x7f8a70067030 2024-08-17T02:22:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a7644c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700ced40 con 0x7f8a700cfb00 2024-08-17T02:22:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a75c4b700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8a70158cc0 0x7f8a701590f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50976/0 (socket says 172.21.15.100:50976) 2024-08-17T02:22:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a75c4b700 1 -- 172.21.15.100:0/2702795457 learned_addr learned my addr 172.21.15.100:0/2702795457 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:19.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a74c49700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a700cfb00 0x7f8a70067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998064628 0 0) 0x7f8a700cf670 con 0x7f8a70158cc0 2024-08-17T02:22:19.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a5c003570 con 0x7f8a70158cc0 2024-08-17T02:22:19.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1429958301 0 0) 0x7f8a5c003570 con 0x7f8a70158cc0 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a700cfb00 msgr2=0x7f8a70067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a700cfb00 0x7f8a70067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 >> v1:172.21.15.123:6789/0 conn(0x7f8a70067030 legacy=0x7f8a700cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700cf020 con 0x7f8a70158cc0 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f8a680017f0 con 0x7f8a70158cc0 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 >> v1:172.21.15.100:6789/0 conn(0x7f8a70158cc0 legacy=0x7f8a701590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a700cfb00 0x7f8a5c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5c003fd0 0x7f8a5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a5c004d50 0x7f8a5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700ced40 con 0x7f8a5c003fd0 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c00bb40 con 0x7f8a5c004d50 2024-08-17T02:22:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c00bd00 con 0x7f8a700cfb00 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a74c49700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5c003fd0 0x7f8a5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a7544a700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a700cfb00 0x7f8a5c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.452+0000 7f8a75c4b700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a5c004d50 0x7f8a5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a74c49700 1 -- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a700cfb00 msgr2=0x7f8a5c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a74c49700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a700cfb00 0x7f8a5c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a74c49700 1 -- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a5c004d50 msgr2=0x7f8a5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a74c49700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a5c004d50 0x7f8a5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a74c49700 1 -- 172.21.15.100:0/2702795457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c003570 con 0x7f8a5c003fd0 2024-08-17T02:22:19.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a75c4b700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a5c004d50 0x7f8a5c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:19.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a74c49700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5c003fd0 0x7f8a5c004810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009c00 tx=0x7f8a6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119579da92905c59 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00e710 con 0x7f8a5c003fd0 2024-08-17T02:22:19.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c00e870 con 0x7f8a5c003fd0 2024-08-17T02:22:19.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a67fff700 1 -- 172.21.15.100:0/2702795457 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00ec30 con 0x7f8a5c003fd0 2024-08-17T02:22:19.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- 172.21.15.100:0/2702795457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5c003fd0 msgr2=0x7f8a5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 --2- 172.21.15.100:0/2702795457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5c003fd0 0x7f8a5c004810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009c00 tx=0x7f8a6c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a75c4b700 1 -- 172.21.15.100:0/2702795457 reap_dead start 2024-08-17T02:22:19.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- 172.21.15.100:0/2702795457 shutdown_connections 2024-08-17T02:22:19.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- 172.21.15.100:0/2702795457 >> 172.21.15.100:0/2702795457 conn(0x7f8a700d0b50 msgr2=0x7f8a700ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:19.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- 172.21.15.100:0/2702795457 shutdown_connections 2024-08-17T02:22:19.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- 172.21.15.100:0/2702795457 wait complete. 2024-08-17T02:22:19.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 Processor -- start 2024-08-17T02:22:19.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- start start 2024-08-17T02:22:19.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a5c003fd0 0x7f8a701439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a70151c80 0x7f8a701696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a701520f0 0x7f8a70169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70151210 con 0x7f8a70151c80 2024-08-17T02:22:19.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7016a250 con 0x7f8a701520f0 2024-08-17T02:22:19.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a7644c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7016a530 con 0x7f8a5c003fd0 2024-08-17T02:22:19.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.456+0000 7f8a74c49700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a70151c80 0x7f8a701696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7544a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a5c003fd0 0x7f8a701439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a74c49700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a70151c80 0x7f8a701696e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52692/0 (socket says 172.21.15.100:52692) 2024-08-17T02:22:19.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a74c49700 1 -- 172.21.15.100:0/3393292909 learned_addr learned my addr 172.21.15.100:0/3393292909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:19.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a75c4b700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a701520f0 0x7f8a70169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7544a700 1 -- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a701520f0 msgr2=0x7f8a70169c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7544a700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a701520f0 0x7f8a70169c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7544a700 1 -- 172.21.15.100:0/3393292909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a70151c80 msgr2=0x7f8a701696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7544a700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a70151c80 0x7f8a701696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7544a700 1 -- 172.21.15.100:0/3393292909 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70151480 con 0x7f8a5c003fd0 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a74c49700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a70151c80 0x7f8a701696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7544a700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a5c003fd0 0x7f8a701439d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a600029b0 tx=0x7f8a60007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a657fa700 1 -- 172.21.15.100:0/3393292909 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60002ea0 con 0x7f8a5c003fd0 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a657fa700 1 -- 172.21.15.100:0/3393292909 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000f8d0 con 0x7f8a5c003fd0 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a70151760 con 0x7f8a5c003fd0 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a657fa700 1 -- 172.21.15.100:0/3393292909 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000fc70 con 0x7f8a5c003fd0 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a70143f10 con 0x7f8a5c003fd0 2024-08-17T02:22:19.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.460+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a701519a0 con 0x7f8a5c003fd0 2024-08-17T02:22:19.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.464+0000 7f8a657fa700 1 -- 172.21.15.100:0/3393292909 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8a6001c400 con 0x7f8a5c003fd0 2024-08-17T02:22:19.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.464+0000 7f8a657fa700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8a54060890 0x7f8a54062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.464+0000 7f8a657fa700 1 -- 172.21.15.100:0/3393292909 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a6007d3f0 con 0x7f8a5c003fd0 2024-08-17T02:22:19.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.464+0000 7f8a74c49700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8a54060890 0x7f8a54062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.464+0000 7f8a74c49700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8a54060890 0x7f8a54062d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a700891d0 tx=0x7f8a6c01a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.472+0000 7f8a657fa700 1 -- 172.21.15.100:0/3393292909 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8a6004d900 con 0x7f8a5c003fd0 2024-08-17T02:22:19.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.632+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a7016a670 con 0x7f8a5c003fd0 2024-08-17T02:22:19.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.632+0000 7f8a657fa700 1 -- 172.21.15.100:0/3393292909 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a7016a670 con 0x7f8a5c003fd0 2024-08-17T02:22:19.637 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:22:19.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8a54060890 msgr2=0x7f8a54062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8a54060890 0x7f8a54062d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a700891d0 tx=0x7f8a6c01a000 comp rx=0 tx=0).stop 2024-08-17T02:22:19.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a5c003fd0 msgr2=0x7f8a701439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a5c003fd0 0x7f8a701439d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a600029b0 tx=0x7f8a60007d10 comp rx=0 tx=0).stop 2024-08-17T02:22:19.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 shutdown_connections 2024-08-17T02:22:19.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8a54060890 0x7f8a54062d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a701520f0 0x7f8a70169c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a70151c80 0x7f8a701696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 --2- 172.21.15.100:0/3393292909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8a5c003fd0 0x7f8a701439d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 >> 172.21.15.100:0/3393292909 conn(0x7f8a700d0b50 msgr2=0x7f8a700b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:19.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 shutdown_connections 2024-08-17T02:22:19.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.636+0000 7f8a7644c700 1 -- 172.21.15.100:0/3393292909 wait complete. 2024-08-17T02:22:19.651 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:22:19.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff017c04700 1 Processor -- start 2024-08-17T02:22:19.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff017c04700 1 -- start start 2024-08-17T02:22:19.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff017c04700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0100b5750 0x7ff0100b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff017c04700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff010159ce0 con 0x7ff0100b2210 2024-08-17T02:22:19.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff017c04700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff010159fc0 con 0x7ff0100cc840 2024-08-17T02:22:19.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff017c04700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100af570 con 0x7ff0100b5750 2024-08-17T02:22:19.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff01519f700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff0100b2210 0x7ff0100b5040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50990/0 (socket says 172.21.15.100:50990) 2024-08-17T02:22:19.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff01519f700 1 -- 172.21.15.100:0/761598290 learned_addr learned my addr 172.21.15.100:0/761598290 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:19.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff0161a1700 1 --2- 172.21.15.100:0/761598290 >> v2:172.21.15.123:3300/0 conn(0x7ff0100b5750 0x7ff0100b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.760+0000 7ff01499e700 1 -- 172.21.15.100:0/761598290 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740776199 0 0) 0x7ff010159ce0 con 0x7ff0100b2210 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff01499e700 1 -- 172.21.15.100:0/761598290 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feffc003570 con 0x7ff0100b2210 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff0161a1700 1 -- 172.21.15.100:0/761598290 >> v1:172.21.15.123:6789/0 conn(0x7ff0100cc840 legacy=0x7ff0100ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff0161a1700 1 -- 172.21.15.100:0/761598290 >> v1:172.21.15.100:6789/0 conn(0x7ff0100b2210 legacy=0x7ff0100b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff0161a1700 1 -- 172.21.15.100:0/761598290 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01015a2a0 con 0x7ff0100b5750 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff0161a1700 1 --2- 172.21.15.100:0/761598290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100b5750 0x7ff0100b2670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff008009e60 tx=0x7ff008009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0266f0e75f2f582 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff01499e700 1 -- 172.21.15.100:0/761598290 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800e070 con 0x7ff0100b5750 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff01499e700 1 -- 172.21.15.100:0/761598290 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00800bb60 con 0x7ff0100b5750 2024-08-17T02:22:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff01499e700 1 -- 172.21.15.100:0/761598290 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008012430 con 0x7ff0100b5750 2024-08-17T02:22:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff017c04700 1 -- 172.21.15.100:0/761598290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100b5750 msgr2=0x7ff0100b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff017c04700 1 --2- 172.21.15.100:0/761598290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100b5750 0x7ff0100b2670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff008009e60 tx=0x7ff008009780 comp rx=0 tx=0).stop 2024-08-17T02:22:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff017c04700 1 -- 172.21.15.100:0/761598290 shutdown_connections 2024-08-17T02:22:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff017c04700 1 --2- 172.21.15.100:0/761598290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100b5750 0x7ff0100b2670 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff017c04700 1 -- 172.21.15.100:0/761598290 >> 172.21.15.100:0/761598290 conn(0x7ff0100682c0 msgr2=0x7ff0100686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff017c04700 1 -- 172.21.15.100:0/761598290 shutdown_connections 2024-08-17T02:22:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff017c04700 1 -- 172.21.15.100:0/761598290 wait complete. 2024-08-17T02:22:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.764+0000 7ff017c04700 1 Processor -- start 2024-08-17T02:22:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 -- start start 2024-08-17T02:22:19.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0100b2210 0x7ff010153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0100b5750 0x7ff0100ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100cc840 0x7ff0100cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010169aa0 con 0x7ff0100b2210 2024-08-17T02:22:19.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010169d80 con 0x7ff0100cc840 2024-08-17T02:22:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01016a060 con 0x7ff0100b5750 2024-08-17T02:22:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0159a0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0100b2210 0x7ff010153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff01519f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0100b5750 0x7ff0100ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0161a1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100cc840 0x7ff0100cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0159a0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0100b2210 0x7ff010153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52702/0 (socket says 172.21.15.100:52702) 2024-08-17T02:22:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0159a0700 1 -- 172.21.15.100:0/4218853011 learned_addr learned my addr 172.21.15.100:0/4218853011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:19.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0161a1700 1 -- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0100b5750 msgr2=0x7ff0100ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0161a1700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0100b5750 0x7ff0100ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0161a1700 1 -- 172.21.15.100:0/4218853011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0100b2210 msgr2=0x7ff010153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0161a1700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0100b2210 0x7ff010153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0161a1700 1 -- 172.21.15.100:0/4218853011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008009a20 con 0x7ff0100cc840 2024-08-17T02:22:19.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff01519f700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0100b5750 0x7ff0100ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:19.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0159a0700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0100b2210 0x7ff010153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:19.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff0161a1700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100cc840 0x7ff0100cebf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff008016000 tx=0x7ff008019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff001ffb700 1 -- 172.21.15.100:0/4218853011 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008027470 con 0x7ff0100cc840 2024-08-17T02:22:19.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff001ffb700 1 -- 172.21.15.100:0/4218853011 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0080275f0 con 0x7ff0100cc840 2024-08-17T02:22:19.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01016a340 con 0x7ff0100cc840 2024-08-17T02:22:19.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff001ffb700 1 -- 172.21.15.100:0/4218853011 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080279b0 con 0x7ff0100cc840 2024-08-17T02:22:19.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.768+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff010153750 con 0x7ff0100cc840 2024-08-17T02:22:19.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.772+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01016a1a0 con 0x7ff0100cc840 2024-08-17T02:22:19.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.772+0000 7ff001ffb700 1 -- 172.21.15.100:0/4218853011 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff008024070 con 0x7ff0100cc840 2024-08-17T02:22:19.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.772+0000 7ff001ffb700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feff4060860 0x7feff4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:19.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.772+0000 7ff001ffb700 1 -- 172.21.15.100:0/4218853011 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff00807f280 con 0x7ff0100cc840 2024-08-17T02:22:19.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.772+0000 7ff0159a0700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feff4060860 0x7feff4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:19.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.776+0000 7ff0159a0700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feff4060860 0x7feff4062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0100cf3a0 tx=0x7ff004009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:19.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.784+0000 7ff001ffb700 1 -- 172.21.15.100:0/4218853011 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff01016a1a0 con 0x7ff0100cc840 2024-08-17T02:22:19.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.948+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff01016a1a0 con 0x7ff0100cc840 2024-08-17T02:22:19.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.952+0000 7ff001ffb700 1 -- 172.21.15.100:0/4218853011 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff01016a1a0 con 0x7ff0100cc840 2024-08-17T02:22:19.953 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:22:19.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feff4060860 msgr2=0x7feff4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feff4060860 0x7feff4062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0100cf3a0 tx=0x7ff004009040 comp rx=0 tx=0).stop 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100cc840 msgr2=0x7ff0100cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100cc840 0x7ff0100cebf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff008016000 tx=0x7ff008019000 comp rx=0 tx=0).stop 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 shutdown_connections 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feff4060860 0x7feff4062d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0100cc840 0x7ff0100cebf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0100b5750 0x7ff0100ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 --2- 172.21.15.100:0/4218853011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0100b2210 0x7ff010153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 >> 172.21.15.100:0/4218853011 conn(0x7ff0100682c0 msgr2=0x7ff0100cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 shutdown_connections 2024-08-17T02:22:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:19.956+0000 7ff017c04700 1 -- 172.21.15.100:0/4218853011 wait complete. 2024-08-17T02:22:19.972 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:20.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafb4a69700 1 Processor -- start 2024-08-17T02:22:20.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafb4a69700 1 -- start start 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafb4a69700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fafb0156db0 0x7fafb00680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafb4a69700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0067690 0x7fafb006b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafb4a69700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb00b2450 con 0x7fafb0157180 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafb4a69700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb0067e00 con 0x7fafb0156db0 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafb4a69700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb006af00 con 0x7fafb0067690 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafaeffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0067690 0x7fafb006b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafaffff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fafb0157180 0x7fafb015aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51006/0 (socket says 172.21.15.100:51006) 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafaffff700 1 -- 172.21.15.100:0/3666850244 learned_addr learned my addr 172.21.15.100:0/3666850244 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafaf7fe700 1 --2- 172.21.15.100:0/3666850244 >> v2:172.21.15.123:3300/0 conn(0x7fafb0156db0 0x7fafb00680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafaeffd700 1 -- 172.21.15.100:0/3666850244 >> v2:172.21.15.123:3300/0 conn(0x7fafb0156db0 msgr2=0x7fafb00680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:20.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.076+0000 7fafae7fc700 1 -- 172.21.15.100:0/3666850244 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132600315 0 0) 0x7fafb00b2450 con 0x7fafb0157180 2024-08-17T02:22:20.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafaeffd700 1 --2- 172.21.15.100:0/3666850244 >> v2:172.21.15.123:3300/0 conn(0x7fafb0156db0 0x7fafb00680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafaeffd700 1 -- 172.21.15.100:0/3666850244 >> v1:172.21.15.100:6789/0 conn(0x7fafb0157180 legacy=0x7fafb015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafaeffd700 1 -- 172.21.15.100:0/3666850244 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0067b20 con 0x7fafb0067690 2024-08-17T02:22:20.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafaeffd700 1 --2- 172.21.15.100:0/3666850244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0067690 0x7fafb006b370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faf980098e0 tx=0x7faf980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991573f23a8c0ad server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafae7fc700 1 -- 172.21.15.100:0/3666850244 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800b640 con 0x7fafb0067690 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafae7fc700 1 -- 172.21.15.100:0/3666850244 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9800b7a0 con 0x7fafb0067690 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafae7fc700 1 -- 172.21.15.100:0/3666850244 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800bb40 con 0x7fafb0067690 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 -- 172.21.15.100:0/3666850244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0067690 msgr2=0x7fafb006b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 --2- 172.21.15.100:0/3666850244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0067690 0x7fafb006b370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faf980098e0 tx=0x7faf980091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 -- 172.21.15.100:0/3666850244 shutdown_connections 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 --2- 172.21.15.100:0/3666850244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0067690 0x7fafb006b370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 --2- 172.21.15.100:0/3666850244 >> v2:172.21.15.123:3300/0 conn(0x7fafb0156db0 0x7fafb00680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 -- 172.21.15.100:0/3666850244 >> 172.21.15.100:0/3666850244 conn(0x7fafb00ccdb0 msgr2=0x7fafb00cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 -- 172.21.15.100:0/3666850244 shutdown_connections 2024-08-17T02:22:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 -- 172.21.15.100:0/3666850244 wait complete. 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 Processor -- start 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.080+0000 7fafb4a69700 1 -- start start 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafb0067690 0x7fafb0146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0156db0 0x7fafb00b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb0157180 0x7fafb0159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00b02a0 con 0x7fafb0157180 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00b0580 con 0x7fafb0067690 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00b0860 con 0x7fafb0156db0 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb0157180 0x7fafb0159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafb0067690 0x7fafb0146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaeffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0156db0 0x7fafb00b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafb0067690 0x7fafb0146850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:38020/0 (socket says 172.21.15.100:38020) 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaf7fe700 1 -- 172.21.15.100:0/1555801171 learned_addr learned my addr 172.21.15.100:0/1555801171 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb0157180 0x7fafb0159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52706/0 (socket says 172.21.15.100:52706) 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaffff700 1 -- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0156db0 msgr2=0x7fafb00b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaffff700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0156db0 0x7fafb00b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaffff700 1 -- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafb0067690 msgr2=0x7fafb0146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaffff700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafb0067690 0x7fafb0146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaffff700 1 -- 172.21.15.100:0/1555801171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf98009420 con 0x7fafb0157180 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaeffd700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0156db0 0x7fafb00b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafaffff700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb0157180 0x7fafb0159500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fafa800cb90 tx=0x7fafa80018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafa77fe700 1 -- 172.21.15.100:0/1555801171 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa800a780 con 0x7fafb0157180 2024-08-17T02:22:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafa77fe700 1 -- 172.21.15.100:0/1555801171 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa800a8e0 con 0x7fafb0157180 2024-08-17T02:22:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb00b0b40 con 0x7fafb0157180 2024-08-17T02:22:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafa77fe700 1 -- 172.21.15.100:0/1555801171 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa800aca0 con 0x7fafb0157180 2024-08-17T02:22:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb0146d90 con 0x7fafb0157180 2024-08-17T02:22:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.084+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb00638f0 con 0x7fafb0157180 2024-08-17T02:22:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.088+0000 7fafa77fe700 1 -- 172.21.15.100:0/1555801171 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fafa8007300 con 0x7fafb0157180 2024-08-17T02:22:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.088+0000 7fafa77fe700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faf94060860 0x7faf94062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.088+0000 7fafa77fe700 1 -- 172.21.15.100:0/1555801171 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fafa807cd90 con 0x7fafb0157180 2024-08-17T02:22:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.088+0000 7fafaf7fe700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faf94060860 0x7faf94062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.088+0000 7fafaf7fe700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faf94060860 0x7faf94062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009e70 tx=0x7fafa00091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.092+0000 7fafa77fe700 1 -- 172.21.15.100:0/1555801171 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fafa804d260 con 0x7fafb0157180 2024-08-17T02:22:20.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.288+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafb00cdaa0 con 0x7fafb0157180 2024-08-17T02:22:20.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.288+0000 7fafa77fe700 1 -- 172.21.15.100:0/1555801171 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fafa8050880 con 0x7fafb0157180 2024-08-17T02:22:20.293 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:20.293 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:04.459540+0000","modified":"2024-08-17T02:22:07.136998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faf94060860 msgr2=0x7faf94062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faf94060860 0x7faf94062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009e70 tx=0x7fafa00091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb0157180 msgr2=0x7fafb0159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb0157180 0x7fafb0159500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fafa800cb90 tx=0x7fafa80018f0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 shutdown_connections 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7faf94060860 0x7faf94062d10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb0157180 0x7fafb0159500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafb0156db0 0x7fafb00b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 --2- 172.21.15.100:0/1555801171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafb0067690 0x7fafb0146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 >> 172.21.15.100:0/1555801171 conn(0x7fafb00ccdb0 msgr2=0x7fafb0069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:20.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 shutdown_connections 2024-08-17T02:22:20.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.292+0000 7fafb4a69700 1 -- 172.21.15.100:0/1555801171 wait complete. 2024-08-17T02:22:20.296 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-08-17T02:22:20.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:20.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 Processor -- start 2024-08-17T02:22:20.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- start start 2024-08-17T02:22:20.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f518815d5b0 0x7f51880d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51880cf190 con 0x7f51880d0010 2024-08-17T02:22:20.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51880d09a0 con 0x7f5188159040 2024-08-17T02:22:20.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518815d9f0 con 0x7f518815d5b0 2024-08-17T02:22:20.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f5187fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f51880d0010 0x7f518815cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51008/0 (socket says 172.21.15.100:51008) 2024-08-17T02:22:20.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f5187fff700 1 -- 172.21.15.100:0/3564047318 learned_addr learned my addr 172.21.15.100:0/3564047318 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:20.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518d7eb700 1 --2- 172.21.15.100:0/3564047318 >> v2:172.21.15.123:3300/0 conn(0x7f518815d5b0 0x7f51880d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f51877fe700 1 -- 172.21.15.100:0/3564047318 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892293068 0 0) 0x7f51880cf190 con 0x7f51880d0010 2024-08-17T02:22:20.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f51877fe700 1 -- 172.21.15.100:0/3564047318 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5174003570 con 0x7f51880d0010 2024-08-17T02:22:20.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518d7eb700 1 -- 172.21.15.100:0/3564047318 >> v1:172.21.15.123:6789/0 conn(0x7f5188159040 legacy=0x7f51880cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518d7eb700 1 -- 172.21.15.100:0/3564047318 >> v1:172.21.15.100:6789/0 conn(0x7f51880d0010 legacy=0x7f518815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518d7eb700 1 -- 172.21.15.100:0/3564047318 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880ce5e0 con 0x7f518815d5b0 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518d7eb700 1 --2- 172.21.15.100:0/3564047318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 0x7f51880d0460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f51800098e0 tx=0x7f51800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae0b446c134cebf server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f51877fe700 1 -- 172.21.15.100:0/3564047318 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518000b640 con 0x7f518815d5b0 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f51877fe700 1 -- 172.21.15.100:0/3564047318 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518000b7a0 con 0x7f518815d5b0 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f51877fe700 1 -- 172.21.15.100:0/3564047318 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518000bb20 con 0x7f518815d5b0 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- 172.21.15.100:0/3564047318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 msgr2=0x7f51880d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 --2- 172.21.15.100:0/3564047318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 0x7f51880d0460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f51800098e0 tx=0x7f51800091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- 172.21.15.100:0/3564047318 shutdown_connections 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 --2- 172.21.15.100:0/3564047318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 0x7f51880d0460 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- 172.21.15.100:0/3564047318 >> 172.21.15.100:0/3564047318 conn(0x7f5188068250 msgr2=0x7f5188068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- 172.21.15.100:0/3564047318 shutdown_connections 2024-08-17T02:22:20.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.412+0000 7f518dfec700 1 -- 172.21.15.100:0/3564047318 wait complete. 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 Processor -- start 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 -- start start 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51880d0010 0x7f5188151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5188159040 0x7f51880ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 0x7f51880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51880b1ce0 con 0x7f5188159040 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51880b10d0 con 0x7f518815d5b0 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51880b13b0 con 0x7f51880d0010 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5188159040 0x7f51880ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5188159040 0x7f51880ae160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52710/0 (socket says 172.21.15.100:52710) 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 -- 172.21.15.100:0/1078256083 learned_addr learned my addr 172.21.15.100:0/1078256083 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518cfea700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51880d0010 0x7f5188151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518d7eb700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 0x7f51880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 -- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51880d0010 msgr2=0x7f5188151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51880d0010 0x7f5188151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 -- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 msgr2=0x7f51880b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 0x7f51880b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 -- 172.21.15.100:0/1078256083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5180009420 con 0x7f5188159040 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5187fff700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5188159040 0x7f51880ae160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f517000c590 tx=0x7f5170007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5184ff9700 1 -- 172.21.15.100:0/1078256083 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170005b30 con 0x7f5188159040 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5184ff9700 1 -- 172.21.15.100:0/1078256083 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170005cb0 con 0x7f5188159040 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51880b1690 con 0x7f5188159040 2024-08-17T02:22:20.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f5184ff9700 1 -- 172.21.15.100:0/1078256083 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170007320 con 0x7f5188159040 2024-08-17T02:22:20.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.416+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51880ae6a0 con 0x7f5188159040 2024-08-17T02:22:20.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.420+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51880b1e50 con 0x7f5188159040 2024-08-17T02:22:20.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.420+0000 7f5184ff9700 1 -- 172.21.15.100:0/1078256083 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f517001b3b0 con 0x7f5188159040 2024-08-17T02:22:20.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.420+0000 7f5184ff9700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5168060860 0x7f5168062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.420+0000 7f5184ff9700 1 -- 172.21.15.100:0/1078256083 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f517007c000 con 0x7f5188159040 2024-08-17T02:22:20.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.420+0000 7f518cfea700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5168060860 0x7f5168062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.420+0000 7f518cfea700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5168060860 0x7f5168062d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5178007fd0 tx=0x7f5178001630 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.428+0000 7f5184ff9700 1 -- 172.21.15.100:0/1078256083 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f517004c450 con 0x7f5188159040 2024-08-17T02:22:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.592+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51880643a0 con 0x7f5188159040 2024-08-17T02:22:20.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.592+0000 7f5184ff9700 1 -- 172.21.15.100:0/1078256083 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f517004fa70 con 0x7f5188159040 2024-08-17T02:22:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:20.596 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5168060860 msgr2=0x7f5168062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5168060860 0x7f5168062d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5178007fd0 tx=0x7f5178001630 comp rx=0 tx=0).stop 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5188159040 msgr2=0x7f51880ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5188159040 0x7f51880ae160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f517000c590 tx=0x7f5170007e00 comp rx=0 tx=0).stop 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 shutdown_connections 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5168060860 0x7f5168062d10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f518815d5b0 0x7f51880b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5188159040 0x7f51880ae160 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 --2- 172.21.15.100:0/1078256083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51880d0010 0x7f5188151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 >> 172.21.15.100:0/1078256083 conn(0x7f5188068250 msgr2=0x7f518815cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:20.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 shutdown_connections 2024-08-17T02:22:20.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.596+0000 7f518dfec700 1 -- 172.21.15.100:0/1078256083 wait complete. 2024-08-17T02:22:20.609 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:20.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 Processor -- start 2024-08-17T02:22:20.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- start start 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a50140a90 0x7f3a500aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3a50158090 0x7f3a50067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a50069d90 con 0x7f3a501406c0 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a50069000 con 0x7f3a50140a90 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a500af740 con 0x7f3a50158090 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a55119700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a50140a90 0x7f3a500aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a55119700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a50140a90 0x7f3a500aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52714/0 (socket says 172.21.15.100:52714) 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a55119700 1 -- 172.21.15.100:0/1851515285 learned_addr learned my addr 172.21.15.100:0/1851515285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5611b700 1 --2- 172.21.15.100:0/1851515285 >> v2:172.21.15.123:3300/0 conn(0x7f3a50158090 0x7f3a50067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a55119700 1 -- 172.21.15.100:0/1851515285 >> v2:172.21.15.123:3300/0 conn(0x7f3a50158090 msgr2=0x7f3a50067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:20.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a55119700 1 --2- 172.21.15.100:0/1851515285 >> v2:172.21.15.123:3300/0 conn(0x7f3a50158090 0x7f3a50067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a55119700 1 -- 172.21.15.100:0/1851515285 >> v1:172.21.15.123:6789/0 conn(0x7f3a501406c0 legacy=0x7f3a501585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:20.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a55119700 1 -- 172.21.15.100:0/1851515285 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50069500 con 0x7f3a50140a90 2024-08-17T02:22:20.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a55119700 1 --2- 172.21.15.100:0/1851515285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a50140a90 0x7f3a500aec30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3a501584f0 tx=0x7f3a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8391b30796f7e45 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a54918700 1 -- 172.21.15.100:0/1851515285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000b7e0 con 0x7f3a50140a90 2024-08-17T02:22:20.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a54918700 1 -- 172.21.15.100:0/1851515285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4000b940 con 0x7f3a50140a90 2024-08-17T02:22:20.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a54918700 1 -- 172.21.15.100:0/1851515285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000bce0 con 0x7f3a50140a90 2024-08-17T02:22:20.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- 172.21.15.100:0/1851515285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a50140a90 msgr2=0x7f3a500aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1851515285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a50140a90 0x7f3a500aec30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3a501584f0 tx=0x7f3a40009250 comp rx=0 tx=0).stop 2024-08-17T02:22:20.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- 172.21.15.100:0/1851515285 shutdown_connections 2024-08-17T02:22:20.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1851515285 >> v2:172.21.15.123:3300/0 conn(0x7f3a50158090 0x7f3a50067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1851515285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a50140a90 0x7f3a500aec30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- 172.21.15.100:0/1851515285 >> 172.21.15.100:0/1851515285 conn(0x7f3a500ccdb0 msgr2=0x7f3a500cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:20.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- 172.21.15.100:0/1851515285 shutdown_connections 2024-08-17T02:22:20.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.716+0000 7f3a5691c700 1 -- 172.21.15.100:0/1851515285 wait complete. 2024-08-17T02:22:20.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5691c700 1 Processor -- start 2024-08-17T02:22:20.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5691c700 1 -- start start 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5691c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a501406c0 0x7f3a5016af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5691c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a50140a90 0x7f3a500cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5691c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a50158090 0x7f3a50136520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5691c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500b2060 con 0x7f3a501406c0 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5691c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5016aaf0 con 0x7f3a50140a90 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5691c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500b1aa0 con 0x7f3a50158090 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a501406c0 0x7f3a5016af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a501406c0 0x7f3a5016af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52728/0 (socket says 172.21.15.100:52728) 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 -- 172.21.15.100:0/1965807721 learned_addr learned my addr 172.21.15.100:0/1965807721 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5611b700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a50158090 0x7f3a50136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a55119700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a50140a90 0x7f3a500cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 -- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a50158090 msgr2=0x7f3a50136520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a50158090 0x7f3a50136520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 -- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a50140a90 msgr2=0x7f3a500cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a50140a90 0x7f3a500cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 -- 172.21.15.100:0/1965807721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a400094f0 con 0x7f3a501406c0 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a55119700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a50140a90 0x7f3a500cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a5591a700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a501406c0 0x7f3a5016af80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c980 tx=0x7f3a48002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.720+0000 7f3a45ffb700 1 -- 172.21.15.100:0/1965807721 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800ce70 con 0x7f3a501406c0 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a45ffb700 1 -- 172.21.15.100:0/1965807721 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4800a7a0 con 0x7f3a501406c0 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a500b1d80 con 0x7f3a501406c0 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a45ffb700 1 -- 172.21.15.100:0/1965807721 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800ab60 con 0x7f3a501406c0 2024-08-17T02:22:20.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5016b4c0 con 0x7f3a501406c0 2024-08-17T02:22:20.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a500638f0 con 0x7f3a501406c0 2024-08-17T02:22:20.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a45ffb700 1 -- 172.21.15.100:0/1965807721 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3a48012420 con 0x7f3a501406c0 2024-08-17T02:22:20.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a45ffb700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3a34060860 0x7f3a34062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:20.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a45ffb700 1 -- 172.21.15.100:0/1965807721 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a4807d030 con 0x7f3a501406c0 2024-08-17T02:22:20.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.724+0000 7f3a55119700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3a34060860 0x7f3a34062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:20.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.728+0000 7f3a55119700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3a34060860 0x7f3a34062d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a50140f10 tx=0x7f3a40016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.736+0000 7f3a45ffb700 1 -- 172.21.15.100:0/1965807721 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3a4804d480 con 0x7f3a501406c0 2024-08-17T02:22:20.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.932+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a5015a830 con 0x7f3a501406c0 2024-08-17T02:22:20.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.932+0000 7f3a45ffb700 1 -- 172.21.15.100:0/1965807721 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f3a48050aa0 con 0x7f3a501406c0 2024-08-17T02:22:20.935 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:20.935 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:04.459540+0000","modified":"2024-08-17T02:22:07.136998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:22:20.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3a34060860 msgr2=0x7f3a34062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3a34060860 0x7f3a34062d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a50140f10 tx=0x7f3a40016000 comp rx=0 tx=0).stop 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a501406c0 msgr2=0x7f3a5016af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a501406c0 0x7f3a5016af80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c980 tx=0x7f3a48002760 comp rx=0 tx=0).stop 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 shutdown_connections 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3a34060860 0x7f3a34062d10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a50158090 0x7f3a50136520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a50140a90 0x7f3a500cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 --2- 172.21.15.100:0/1965807721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a501406c0 0x7f3a5016af80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 >> 172.21.15.100:0/1965807721 conn(0x7f3a500ccdb0 msgr2=0x7f3a500cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 shutdown_connections 2024-08-17T02:22:20.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:20.936+0000 7f3a5691c700 1 -- 172.21.15.100:0/1965807721 wait complete. 2024-08-17T02:22:20.939 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-08-17T02:22:21.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-17T02:22:04.459540+0000', 'modified': '2024-08-17T02:22:07.136998+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6837/2524989904', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2524989904}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2524989904}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-17T02:22:21.352 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-17T02:22:21.352 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-08-17T02:22:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd306316700 1 Processor -- start 2024-08-17T02:22:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd306316700 1 -- start start 2024-08-17T02:22:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd306316700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd3000ae2a0 0x7fd3000ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd306316700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3000b0f10 0x7fd3000a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd306316700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3000a9160 con 0x7fd30015a620 2024-08-17T02:22:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd306316700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3000aff10 con 0x7fd3000ae2a0 2024-08-17T02:22:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd306316700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000b1680 con 0x7fd3000b0f10 2024-08-17T02:22:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2fffff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd30015a620 0x7fd3000adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51024/0 (socket says 172.21.15.100:51024) 2024-08-17T02:22:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2fffff700 1 -- 172.21.15.100:0/582737567 learned_addr learned my addr 172.21.15.100:0/582737567 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/582737567 >> v2:172.21.15.123:3300/0 conn(0x7fd3000ae2a0 0x7fd3000ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd3048b3700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3000b0f10 0x7fd3000a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946722559 0 0) 0x7fd3000a9160 con 0x7fd30015a620 2024-08-17T02:22:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ec003570 con 0x7fd30015a620 2024-08-17T02:22:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2020309153 0 0) 0x7fd2ec003570 con 0x7fd30015a620 2024-08-17T02:22:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3000b0f10 msgr2=0x7fd3000a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3000b0f10 0x7fd3000a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 >> v2:172.21.15.123:3300/0 conn(0x7fd3000ae2a0 msgr2=0x7fd3000ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 --2- 172.21.15.100:0/582737567 >> v2:172.21.15.123:3300/0 conn(0x7fd3000ae2a0 0x7fd3000ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000b1960 con 0x7fd30015a620 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7fd2f0001470 con 0x7fd30015a620 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 >> v1:172.21.15.100:6789/0 conn(0x7fd30015a620 legacy=0x7fd3000adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec003a60 0x7fd2ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd2ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3000b0f10 0x7fd2ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000aff10 con 0x7fd2ec004810 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000b1680 con 0x7fd2ec003a60 2024-08-17T02:22:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec00ba60 con 0x7fd3000b0f10 2024-08-17T02:22:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd2ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.468+0000 7fd2fffff700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec003a60 0x7fd2ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd3048b3700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3000b0f10 0x7fd2ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2f77fe700 1 -- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3000b0f10 msgr2=0x7fd2ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3000b0f10 0x7fd2ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2f77fe700 1 -- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec003a60 msgr2=0x7fd2ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec003a60 0x7fd2ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2f77fe700 1 -- 172.21.15.100:0/582737567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec003570 con 0x7fd2ec004810 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd2ec004c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009c00 tx=0x7fd2e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463eecde816760e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800e710 con 0x7fd2ec004810 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e800e870 con 0x7fd2ec004810 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd2ff7fe700 1 -- 172.21.15.100:0/582737567 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800ec30 con 0x7fd2ec004810 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd306316700 1 -- 172.21.15.100:0/582737567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 msgr2=0x7fd2ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd306316700 1 --2- 172.21.15.100:0/582737567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd2ec004c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009c00 tx=0x7fd2e80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd3048b3700 1 -- 172.21.15.100:0/582737567 reap_dead start 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd306316700 1 -- 172.21.15.100:0/582737567 shutdown_connections 2024-08-17T02:22:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd306316700 1 -- 172.21.15.100:0/582737567 >> 172.21.15.100:0/582737567 conn(0x7fd300067810 msgr2=0x7fd300067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:21.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd306316700 1 -- 172.21.15.100:0/582737567 shutdown_connections 2024-08-17T02:22:21.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.472+0000 7fd306316700 1 -- 172.21.15.100:0/582737567 wait complete. 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 Processor -- start 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 -- start start 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd3000c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd30016a010 0x7fd30016ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd30006e720 0x7fd3000ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000c66c0 con 0x7fd2ec004810 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300069f70 con 0x7fd30016a010 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30016d2c0 con 0x7fd30006e720 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd3000c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd3000c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52744/0 (socket says 172.21.15.100:52744) 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 -- 172.21.15.100:0/4002836681 learned_addr learned my addr 172.21.15.100:0/4002836681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd30016a010 0x7fd30016ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd3048b3700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd30006e720 0x7fd3000ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 -- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd30006e720 msgr2=0x7fd3000ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd30006e720 0x7fd3000ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 -- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd30016a010 msgr2=0x7fd30016ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd30016a010 0x7fd30016ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 -- 172.21.15.100:0/4002836681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30016d5a0 con 0x7fd2ec004810 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fffff700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd3000c6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0000e10 tx=0x7fd2f0001570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fcff9700 1 -- 172.21.15.100:0/4002836681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000ee70 con 0x7fd2ec004810 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fcff9700 1 -- 172.21.15.100:0/4002836681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f000c6d0 con 0x7fd2ec004810 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd2fcff9700 1 -- 172.21.15.100:0/4002836681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000ca90 con 0x7fd2ec004810 2024-08-17T02:22:21.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 -- 172.21.15.100:0/4002836681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30006a1e0 con 0x7fd2ec004810 2024-08-17T02:22:21.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.476+0000 7fd306316700 1 -- 172.21.15.100:0/4002836681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30006eb90 con 0x7fd2ec004810 2024-08-17T02:22:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd2fcff9700 1 -- 172.21.15.100:0/4002836681 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd2f001d460 con 0x7fd2ec004810 2024-08-17T02:22:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd2fcff9700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd2e0060860 0x7fd2e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd2fcff9700 1 -- 172.21.15.100:0/4002836681 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2f007caf0 con 0x7fd2ec004810 2024-08-17T02:22:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd2e0060860 0x7fd2e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3085c5740 1 Processor -- start 2024-08-17T02:22:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3085c5740 1 -- start start 2024-08-17T02:22:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd2e0060860 0x7fd2e0062d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80091e0 tx=0x7fd2e801a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1effae0 0x21331d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eff710 0x2133710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x2136c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3085c5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x20abba0 con 0x1f4a790 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3085c5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x20a7b40 con 0x1eff710 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3085c5740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x206d0a0 con 0x1effae0 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.480+0000 7fd3048b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x2136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd2f77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eff710 0x2133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3048b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x2136c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52748/0 (socket says 172.21.15.100:52748) 2024-08-17T02:22:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3048b3700 1 -- 172.21.15.100:0/3920582939 learned_addr learned my addr 172.21.15.100:0/3920582939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd2fffff700 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1effae0 0x21331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3048b3700 1 -- 172.21.15.100:0/3920582939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1effae0 msgr2=0x21331d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3048b3700 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1effae0 0x21331d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3048b3700 1 -- 172.21.15.100:0/3920582939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eff710 msgr2=0x2133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3048b3700 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eff710 0x2133710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3048b3700 1 -- 172.21.15.100:0/3920582939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f4b9e0 con 0x1f4a790 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eff710 0x2133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3048b3700 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x2136c60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd30006c3a0 tx=0x7fd2f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eff6b3c89b75336 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd2f57fa700 1 -- 172.21.15.100:0/3920582939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f800e030 con 0x1f4a790 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd2f57fa700 1 -- 172.21.15.100:0/3920582939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f800f040 con 0x1f4a790 2024-08-17T02:22:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd2f57fa700 1 -- 172.21.15.100:0/3920582939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8017680 con 0x1f4a790 2024-08-17T02:22:21.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 -- 172.21.15.100:0/3920582939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 msgr2=0x2136c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x2136c60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd30006c3a0 tx=0x7fd2f800b040 comp rx=0 tx=0).stop 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 -- 172.21.15.100:0/3920582939 shutdown_connections 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x2136c60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eff710 0x2133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3920582939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1effae0 0x21331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 -- 172.21.15.100:0/3920582939 >> 172.21.15.100:0/3920582939 conn(0x1effeb0 msgr2=0x21390b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 -- 172.21.15.100:0/3920582939 shutdown_connections 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 -- 172.21.15.100:0/3920582939 wait complete. 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 -- start start 2024-08-17T02:22:21.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.484+0000 7fd3085c5740 1 Processor -- start 2024-08-17T02:22:21.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x2180aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x2180fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x213afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x206cd40 con 0x1f4a790 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x2045970 con 0x1effae0 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x20a83e0 con 0x1eff710 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 10 client.0 _ll_register_callbacks cb 0x2049f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x2180aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3048b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x213afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x2180aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:54482/0 (socket says 172.21.15.100:54482) 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 -- 172.21.15.100:0/3850489749 learned_addr learned my addr 172.21.15.100:0/3850489749 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x2180fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 -- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 msgr2=0x2180fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x2180fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 -- 172.21.15.100:0/3850489749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 msgr2=0x213afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x213afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 -- 172.21.15.100:0/3850489749 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec003570 con 0x1eff710 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3048b3700 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x213afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2fffff700 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x2180aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0001050 tx=0x7fd2f0056040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7086b7e5770d5482 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.123:3301/0 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2de7fc700 1 -- 172.21.15.100:0/3850489749 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0066210 con 0x1eff710 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 -- 172.21.15.100:0/3850489749 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f0d260 con 0x1eff710 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2de7fc700 1 -- 172.21.15.100:0/3850489749 <== mon.2 v2:172.21.15.123:3301/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2f000ce00 con 0x1eff710 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2de7fc700 1 -- 172.21.15.100:0/3850489749 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f0050ea0 con 0x1eff710 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 10 client.4535 fetch_fsmap learned FSMap version 6 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 -- 172.21.15.100:0/3850489749 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x7fd2f8002280 con 0x1eff710 2024-08-17T02:22:21.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2de7fc700 1 -- 172.21.15.100:0/3850489749 <== mon.2 v2:172.21.15.123:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f0d260 con 0x1eff710 2024-08-17T02:22:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2de7fc700 1 -- 172.21.15.100:0/3850489749 <== mon.2 v2:172.21.15.123:3301/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd2f00500f0 con 0x1eff710 2024-08-17T02:22:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 10 client.4535 fetch_fsmap finished waiting for FSMap version 6 2024-08-17T02:22:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 10 client.4535 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-17T02:22:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 20 client.4535 populate_metadata read hostname 'smithi100' 2024-08-17T02:22:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] conn(0x2189c30 0x218c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 4 client.4535 mds_command: new command op to 4270 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-17T02:22:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd3085c5740 1 -- 172.21.15.100:0/3850489749 --> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1f574e0 con 0x2189c30 2024-08-17T02:22:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.488+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] conn(0x2189c30 0x218c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] conn(0x2189c30 0x218c010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967239e0f0a543e7 server_cookie=84ca4cade7b94667 in_seq=0 out_seq=0 2024-08-17T02:22:21.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd2de7fc700 10 client.4535 ms_handle_connect on v2:172.21.15.123:6836/2524989904 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd2de7fc700 1 -- 172.21.15.100:0/3850489749 <== mds.0 v2:172.21.15.123:6836/2524989904 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x1f574e0 con 0x2189c30 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd2de7fc700 10 client.4535 handle_command_reply: tid=0 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 client.4535 shutdown 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 -- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 msgr2=0x2180aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x2180aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0001050 tx=0x7fd2f0056040 comp rx=0 tx=0).stop 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 -- 172.21.15.100:0/3850489749 shutdown_connections 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] conn(0x2189c30 0x218c010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x213afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x2180fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd2de7fc700 0 client.4535 ms_handle_reset on v2:172.21.15.123:6836/2524989904 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 --2- 172.21.15.100:0/3850489749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x2180aa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 -- 172.21.15.100:0/3850489749 >> 172.21.15.100:0/3850489749 conn(0x1effeb0 msgr2=0x2138ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:21.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 -- 172.21.15.100:0/3850489749 shutdown_connections 2024-08-17T02:22:21.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 1 -- 172.21.15.100:0/3850489749 wait complete. 2024-08-17T02:22:21.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.492+0000 7fd3085c5740 20 client.4535 trim_cache size 0 max 16384 2024-08-17T02:22:21.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.512+0000 7fd3085c5740 1 Processor -- start 2024-08-17T02:22:21.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.512+0000 7fd3085c5740 1 -- start start 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x21d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x21d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f7c650 con 0x1f4a790 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x2073d40 con 0x1effae0 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x20a2080 con 0x1eff710 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x21d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x21d55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52764/0 (socket says 172.21.15.100:52764) 2024-08-17T02:22:21.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 -- 172.21.15.100:0/273885612 learned_addr learned my addr 172.21.15.100:0/273885612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd2fffff700 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x21d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 -- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 msgr2=0x21d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x21d4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 -- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 msgr2=0x21d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 -- 172.21.15.100:0/273885612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2075370 con 0x1f4a790 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3048b3700 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x21d55b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x2073fc0 tx=0x7fd2f801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564ca61b2b297592 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd2f5ffb700 1 -- 172.21.15.100:0/273885612 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f800e050 con 0x1f4a790 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd2f5ffb700 1 -- 172.21.15.100:0/273885612 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f8002ed0 con 0x1f4a790 2024-08-17T02:22:21.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd2f5ffb700 1 -- 172.21.15.100:0/273885612 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8027070 con 0x1f4a790 2024-08-17T02:22:21.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 -- 172.21.15.100:0/273885612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 msgr2=0x21d55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x21d55b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x2073fc0 tx=0x7fd2f801d000 comp rx=0 tx=0).stop 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 -- 172.21.15.100:0/273885612 shutdown_connections 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1f4a790 0x21d55b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 --2- 172.21.15.100:0/273885612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eff710 0x21d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 -- 172.21.15.100:0/273885612 >> 172.21.15.100:0/273885612 conn(0x1effeb0 msgr2=0x2139370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 -- 172.21.15.100:0/273885612 shutdown_connections 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.516+0000 7fd3085c5740 1 -- 172.21.15.100:0/273885612 wait complete. 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 1 -- start start 2024-08-17T02:22:21.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 1 Processor -- start 2024-08-17T02:22:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1eff710 0x21d86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f4a790 0x218c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x20aa200 con 0x1eff710 2024-08-17T02:22:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x20a3650 con 0x1effae0 2024-08-17T02:22:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x20a5500 con 0x1f4a790 2024-08-17T02:22:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1eff710 0x21d86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1eff710 0x21d86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52768/0 (socket says 172.21.15.100:52768) 2024-08-17T02:22:21.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 -- 172.21.15.100:0/1169323945 learned_addr learned my addr 172.21.15.100:0/1169323945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:21.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3085c5740 10 client.0 _ll_register_callbacks cb 0x1ecb9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-17T02:22:21.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 -- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f4a790 msgr2=0x218c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3048b3700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f4a790 0x218c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f4a790 0x218c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 -- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 msgr2=0x21d8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:21.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 -- 172.21.15.100:0/1169323945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1f4b9e0 con 0x1eff710 2024-08-17T02:22:21.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd3048b3700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f4a790 0x218c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:21.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:21.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2fffff700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1eff710 0x21d86f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000a040 tx=0x7fd2f001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea81e61c4281126 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.520+0000 7fd2de7fc700 1 -- 172.21.15.100:0/1169323945 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0066210 con 0x1eff710 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2de7fc700 1 -- 172.21.15.100:0/1169323945 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2f0057be0 con 0x1eff710 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 -- 172.21.15.100:0/1169323945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2f800f040 con 0x1eff710 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2de7fc700 1 -- 172.21.15.100:0/1169323945 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f00579b0 con 0x1eff710 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 10 client.4707 fetch_fsmap learned FSMap version 6 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 -- 172.21.15.100:0/1169323945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x205bd80 con 0x1eff710 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2de7fc700 1 -- 172.21.15.100:0/1169323945 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f800f040 con 0x1eff710 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2de7fc700 1 -- 172.21.15.100:0/1169323945 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd2f00752f0 con 0x1eff710 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 10 client.4707 fetch_fsmap finished waiting for FSMap version 6 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 10 client.4707 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 20 client.4707 populate_metadata read hostname 'smithi100' 2024-08-17T02:22:21.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] conn(0x218cc30 0x218d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 4 client.4707 mds_command: new command op to 4270 tid=0[{"prefix": "status"}] 2024-08-17T02:22:21.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 -- 172.21.15.100:0/1169323945 --> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1f7c260 con 0x218cc30 2024-08-17T02:22:21.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] conn(0x218cc30 0x218d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2de7fc700 10 client.4707 ms_handle_connect on v2:172.21.15.123:6836/2524989904 2024-08-17T02:22:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2f77fe700 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] conn(0x218cc30 0x218d000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bda6935d0fecc8 server_cookie=51df3416622dec95 in_seq=0 out_seq=0 2024-08-17T02:22:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2de7fc700 1 -- 172.21.15.100:0/1169323945 <== mds.0 v2:172.21.15.123:6836/2524989904 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x1f7c260 con 0x218cc30 2024-08-17T02:22:21.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2de7fc700 10 client.4707 handle_command_reply: tid=0 2024-08-17T02:22:21.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 client.4707 shutdown 2024-08-17T02:22:21.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 -- 172.21.15.100:0/1169323945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1eff710 msgr2=0x21d86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1eff710 0x21d86f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000a040 tx=0x7fd2f001e000 comp rx=0 tx=0).stop 2024-08-17T02:22:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 -- 172.21.15.100:0/1169323945 shutdown_connections 2024-08-17T02:22:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:6836/2524989904,v1:172.21.15.123:6837/2524989904] conn(0x218cc30 0x218d000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f4a790 0x218c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd2de7fc700 0 client.4707 ms_handle_reset on v2:172.21.15.123:6836/2524989904 2024-08-17T02:22:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1effae0 0x21d8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 --2- 172.21.15.100:0/1169323945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x1eff710 0x21d86f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 -- 172.21.15.100:0/1169323945 >> 172.21.15.100:0/1169323945 conn(0x1effeb0 msgr2=0x217f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 -- 172.21.15.100:0/1169323945 shutdown_connections 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 1 -- 172.21.15.100:0/1169323945 wait complete. 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.524+0000 7fd3085c5740 20 client.4707 trim_cache size 0 max 16384 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "147b3f96-a106-4648-b442-f3eda3a290c0", 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4270, 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-08-17T02:22:21.535 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-08-17T02:22:21.536 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 16.422839867, 2024-08-17T02:22:21.536 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 6, 2024-08-17T02:22:21.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 22, 2024-08-17T02:22:21.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 22, 2024-08-17T02:22:21.536 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 23.081855743999999 2024-08-17T02:22:21.536 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:22:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 -- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd2e0060860 msgr2=0x7fd2e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd2e0060860 0x7fd2e0062d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80091e0 tx=0x7fd2e801a000 comp rx=0 tx=0).stop 2024-08-17T02:22:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 -- 172.21.15.100:0/4002836681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 msgr2=0x7fd3000c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd3000c6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0000e10 tx=0x7fd2f0001570 comp rx=0 tx=0).stop 2024-08-17T02:22:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 -- 172.21.15.100:0/4002836681 shutdown_connections 2024-08-17T02:22:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd2e0060860 0x7fd2e0062d10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd30006e720 0x7fd3000ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd30016a010 0x7fd30016ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 --2- 172.21.15.100:0/4002836681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec004810 0x7fd3000c6180 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 -- 172.21.15.100:0/4002836681 >> 172.21.15.100:0/4002836681 conn(0x7fd300067810 msgr2=0x7fd300153920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 -- 172.21.15.100:0/4002836681 shutdown_connections 2024-08-17T02:22:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.528+0000 7fd306316700 1 -- 172.21.15.100:0/4002836681 wait complete. 2024-08-17T02:22:21.546 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-08-17T02:22:21.572 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-08-17T02:22:21.572 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-08-17T02:22:21.572 INFO:tasks.ceph_fuse:client.0 config is {} 2024-08-17T02:22:21.572 INFO:tasks.ceph_fuse:client.2 config is {} 2024-08-17T02:22:21.573 INFO:tasks.ceph_fuse:client.1 config is {} 2024-08-17T02:22:21.573 INFO:tasks.ceph_fuse:client.3 config is {} 2024-08-17T02:22:21.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:21.573 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-08-17T02:22:21.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:21.581 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-08-17T02:22:21.602 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-08-17T02:22:21.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:22:21.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:21.604 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-08-17T02:22:21.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:21.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-08-17T02:22:21.629 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-08-17T02:22:21.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:22:21.630 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-08-17T02:22:21.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:22:21.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.736+0000 7f24aa7ba700 1 Processor -- start 2024-08-17T02:22:21.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.736+0000 7f24aa7ba700 1 -- start start 2024-08-17T02:22:21.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.736+0000 7f24aa7ba700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24a4156f80 0x7f24a40ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.736+0000 7f24aa7ba700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40adf50 con 0x7f24a406f040 2024-08-17T02:22:21.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.736+0000 7f24aa7ba700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40ae920 con 0x7f24a40ace80 2024-08-17T02:22:21.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.736+0000 7f24aa7ba700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a40ad340 con 0x7f24a4156f80 2024-08-17T02:22:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a8d57700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f24a406f040 0x7f24a40aab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51030/0 (socket says 172.21.15.100:51030) 2024-08-17T02:22:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a8d57700 1 -- 172.21.15.100:0/736732174 learned_addr learned my addr 172.21.15.100:0/736732174 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a3fff700 1 --2- 172.21.15.100:0/736732174 >> v2:172.21.15.123:3300/0 conn(0x7f24a4156f80 0x7f24a40ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a2ffd700 1 -- 172.21.15.100:0/736732174 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800226662 0 0) 0x7f24a40adf50 con 0x7f24a406f040 2024-08-17T02:22:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a2ffd700 1 -- 172.21.15.100:0/736732174 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f248c003570 con 0x7f24a406f040 2024-08-17T02:22:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a2ffd700 1 -- 172.21.15.100:0/736732174 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197106690 0 0) 0x7f24a40ae920 con 0x7f24a40ace80 2024-08-17T02:22:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a2ffd700 1 -- 172.21.15.100:0/736732174 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a40adf50 con 0x7f24a40ace80 2024-08-17T02:22:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a3fff700 1 -- 172.21.15.100:0/736732174 >> v1:172.21.15.123:6789/0 conn(0x7f24a40ace80 legacy=0x7f24a4157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a3fff700 1 -- 172.21.15.100:0/736732174 >> v1:172.21.15.100:6789/0 conn(0x7f24a406f040 legacy=0x7f24a40aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a3fff700 1 -- 172.21.15.100:0/736732174 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40ad620 con 0x7f24a4156f80 2024-08-17T02:22:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a3fff700 1 --2- 172.21.15.100:0/736732174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 0x7f24a40ae3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24980098e0 tx=0x7f24980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb983847befa093 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a2ffd700 1 -- 172.21.15.100:0/736732174 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498011630 con 0x7f24a4156f80 2024-08-17T02:22:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a2ffd700 1 -- 172.21.15.100:0/736732174 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498011790 con 0x7f24a4156f80 2024-08-17T02:22:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24a2ffd700 1 -- 172.21.15.100:0/736732174 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498011b10 con 0x7f24a4156f80 2024-08-17T02:22:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24aa7ba700 1 -- 172.21.15.100:0/736732174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 msgr2=0x7f24a40ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24aa7ba700 1 --2- 172.21.15.100:0/736732174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 0x7f24a40ae3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24980098e0 tx=0x7f24980091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24aa7ba700 1 -- 172.21.15.100:0/736732174 shutdown_connections 2024-08-17T02:22:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24aa7ba700 1 --2- 172.21.15.100:0/736732174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 0x7f24a40ae3e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24aa7ba700 1 -- 172.21.15.100:0/736732174 >> 172.21.15.100:0/736732174 conn(0x7f24a4066e00 msgr2=0x7f24a4067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24aa7ba700 1 -- 172.21.15.100:0/736732174 shutdown_connections 2024-08-17T02:22:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.740+0000 7f24aa7ba700 1 -- 172.21.15.100:0/736732174 wait complete. 2024-08-17T02:22:21.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 Processor -- start 2024-08-17T02:22:21.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 -- start start 2024-08-17T02:22:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24a406f040 0x7f24a414dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a40ace80 0x7f24a414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 0x7f24a40b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a414e170 con 0x7f24a40ace80 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a414cc20 con 0x7f24a4156f80 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a414d7a0 con 0x7f24a406f040 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a40ace80 0x7f24a414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a40ace80 0x7f24a414e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52772/0 (socket says 172.21.15.100:52772) 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 -- 172.21.15.100:0/323548618 learned_addr learned my addr 172.21.15.100:0/323548618 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a3fff700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24a406f040 0x7f24a414dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a8d57700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 0x7f24a40b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 -- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24a406f040 msgr2=0x7f24a414dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24a406f040 0x7f24a414dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 -- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 msgr2=0x7f24a40b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 0x7f24a40b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 -- 172.21.15.100:0/323548618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498009420 con 0x7f24a40ace80 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24a37fe700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a40ace80 0x7f24a414e8f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f249400cb10 tx=0x7f24940027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f248bfff700 1 -- 172.21.15.100:0/323548618 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24940040f0 con 0x7f24a40ace80 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f248bfff700 1 -- 172.21.15.100:0/323548618 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249400a7e0 con 0x7f24a40ace80 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a414ce70 con 0x7f24a40ace80 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f248bfff700 1 -- 172.21.15.100:0/323548618 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400aba0 con 0x7f24a40ace80 2024-08-17T02:22:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.744+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a414d430 con 0x7f24a40ace80 2024-08-17T02:22:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.748+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a40645c0 con 0x7f24a40ace80 2024-08-17T02:22:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.748+0000 7f248bfff700 1 -- 172.21.15.100:0/323548618 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2494012420 con 0x7f24a40ace80 2024-08-17T02:22:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.748+0000 7f248bfff700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2490060860 0x7f2490062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.748+0000 7f248bfff700 1 -- 172.21.15.100:0/323548618 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f249407cd10 con 0x7f24a40ace80 2024-08-17T02:22:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.748+0000 7f24a3fff700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2490060860 0x7f2490062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.748+0000 7f24a3fff700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2490060860 0x7f2490062d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24980093f0 tx=0x7f2498017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:21.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.756+0000 7f248bfff700 1 -- 172.21.15.100:0/323548618 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f249404d220 con 0x7f24a40ace80 2024-08-17T02:22:21.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f24a41547c0 con 0x7f24a40ace80 2024-08-17T02:22:21.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f248bfff700 1 -- 172.21.15.100:0/323548618 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2494050840 con 0x7f24a40ace80 2024-08-17T02:22:21.966 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:22:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2490060860 msgr2=0x7f2490062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2490060860 0x7f2490062d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24980093f0 tx=0x7f2498017000 comp rx=0 tx=0).stop 2024-08-17T02:22:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a40ace80 msgr2=0x7f24a414e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a40ace80 0x7f24a414e8f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f249400cb10 tx=0x7f24940027a0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 shutdown_connections 2024-08-17T02:22:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2490060860 0x7f2490062d10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4156f80 0x7f24a40b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a40ace80 0x7f24a414e8f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 --2- 172.21.15.100:0/323548618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24a406f040 0x7f24a414dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:21.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 >> 172.21.15.100:0/323548618 conn(0x7f24a4066e00 msgr2=0x7f24a406afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:21.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 shutdown_connections 2024-08-17T02:22:21.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:21.964+0000 7f24aa7ba700 1 -- 172.21.15.100:0/323548618 wait complete. 2024-08-17T02:22:21.980 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:22:21.980 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:22:21.980 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-08-17T02:22:21.980 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-17T02:22:21.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:21.981 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:22:21.981 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:22:21.981 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:22:21.981 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:22:21.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:21.981 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-08-17T02:22:21.986 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:21.986 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:21.987 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-17T02:22:21.988 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42618sec preferred_lft 42618sec 2024-08-17T02:22:21.988 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-08-17T02:22:21.988 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:21.988 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-17T02:22:21.988 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:7a:f5:6f:c6 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:21.989 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-17T02:22:21.989 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:21.989 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-17T02:22:21.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:21.990 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:21.990 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:21.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-08-17T02:22:21.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-08-17T02:22:21.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-08-17T02:22:21.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-17T02:22:21.990 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:22.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:22.038 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:22:22.049 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-08-17T02:22:22.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:22.051 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-08-17T02:22:22.116 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-08-17T02:22:22.116 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:22:22.116 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-17T02:22:22.116 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:22:22.116 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-17T02:22:22.116 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-17T02:22:22.117 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-17T02:22:22.117 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:22:22.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:22.117 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:22.117 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:22.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:22:22.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:22:22.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:22:22.118 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:22.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:22.159 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-08-17T02:22:22.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:22.166 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-08-17T02:22:22.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:22.217 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:22.217 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:22.217 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-17T02:22:22.217 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-17T02:22:22.218 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:22.247 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-17T02:22:22.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:22.248 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:22.248 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:22.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-17T02:22:22.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-17T02:22:22.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-17T02:22:22.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-17T02:22:22.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-17T02:22:22.248 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:22.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:22.317 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:22.317 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:22.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-08-17T02:22:22.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-08-17T02:22:22.317 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:22.344 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-17T02:22:22.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:22.345 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:22.349 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:22:22.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:22.350 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:22.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:22.417 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:22.418 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:22.432 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:22.434 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:22.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:22.435 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:22.483 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-17T02:22:22.483 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-17T02:22:22.485 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:22.501 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:22.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:22.515 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:22.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:22.515 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:22.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-08-17T02:22:22.560+0000 7f05df6d9380 -1 init, newargv = 0x55929b108a80 newargc=15 2024-08-17T02:22:22.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[30621]: starting ceph client 2024-08-17T02:22:22.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[30621]: starting fuse 2024-08-17T02:22:23.665 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:23.685 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:23.685 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:23.744 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:23.745 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:23.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:23.745 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:23.753 INFO:teuthology.orchestra.run.smithi100.stdout:52 2024-08-17T02:22:23.753 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-17T02:22:23.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-17T02:22:23.753 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-08-17T02:22:23.753 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-08-17T02:22:23.753 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-08-17T02:22:23.753 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-08-17T02:22:23.754 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-08-17T02:22:23.755 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-08-17T02:22:23.842 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.30621.asok 2024-08-17T02:22:24.022 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.30621.asok 2024-08-17T02:22:24.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:24.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.30621.asok status 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "2577e8139c6b8786f9d6d6ea21b8da9244b9260e", 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 17.2.7-1375-g2577e813 (2577e8139c6b8786f9d6d6ea21b8da9244b9260e) quincy (stable)", 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "30621", 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-08-17T02:22:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4719, 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4719 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3079681932 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-08-17T02:22:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-08-17T02:22:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3079681932 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4719 192.168.0.1:0/3079681932", 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/3079681932", 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 6, 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 22, 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-08-17T02:22:24.163 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:22:24.170 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:22:24.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.276+0000 7f3f0b59e700 1 Processor -- start 2024-08-17T02:22:24.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 -- start start 2024-08-17T02:22:24.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f0c155d20 0x7f3f0c0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:24.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0af2b0 con 0x7f3f0c0ad7c0 2024-08-17T02:22:24.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0af590 con 0x7f3f0c06df10 2024-08-17T02:22:24.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c0adc10 con 0x7f3f0c155d20 2024-08-17T02:22:24.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3f0c0ad7c0 0x7f3f0c0aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58764/0 (socket says 172.21.15.100:58764) 2024-08-17T02:22:24.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 -- 172.21.15.100:0/1374723022 learned_addr learned my addr 172.21.15.100:0/1374723022 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:24.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0a59c700 1 --2- 172.21.15.100:0/1374723022 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c155d20 0x7f3f0c0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709765077 0 0) 0x7f3f0c0af2b0 con 0x7f3f0c0ad7c0 2024-08-17T02:22:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef8003570 con 0x7f3f0c0ad7c0 2024-08-17T02:22:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 419535789 0 0) 0x7f3ef8003570 con 0x7f3f0c0ad7c0 2024-08-17T02:22:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c155d20 msgr2=0x7f3f0c0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 --2- 172.21.15.100:0/1374723022 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c155d20 0x7f3f0c0b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 >> v1:172.21.15.123:6789/0 conn(0x7f3f0c06df10 legacy=0x7f3f0c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0af870 con 0x7f3f0c0ad7c0 2024-08-17T02:22:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f3ef4001470 con 0x7f3f0c0ad7c0 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 >> v1:172.21.15.100:6789/0 conn(0x7f3f0c0ad7c0 legacy=0x7f3f0c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef8003a60 0x7f3ef8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef80052e0 0x7f3ef800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0adc10 con 0x7f3ef8004810 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef800c190 con 0x7f3ef80052e0 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef800c350 con 0x7f3ef8003a60 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0a59c700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef8003a60 0x7f3ef8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0ad9d700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef80052e0 0x7f3ef800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 -- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef8003a60 msgr2=0x7f3ef8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef8003a60 0x7f3ef8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 -- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef80052e0 msgr2=0x7f3ef800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef80052e0 0x7f3ef800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 -- 172.21.15.100:0/1374723022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8003570 con 0x7f3ef8004810 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0ad9d700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef80052e0 0x7f3ef800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0ad9d700 1 -- 172.21.15.100:0/1374723022 reap_dead start 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f03fff700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400c3c0 tx=0x7f3ef4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804e3d251fe9a94 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400cc90 con 0x7f3ef8004810 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef400ce10 con 0x7f3ef8004810 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f09d9b700 1 -- 172.21.15.100:0/1374723022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4005b70 con 0x7f3ef8004810 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 -- 172.21.15.100:0/1374723022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 msgr2=0x7f3ef8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400c3c0 tx=0x7f3ef4001e50 comp rx=0 tx=0).stop 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 -- 172.21.15.100:0/1374723022 shutdown_connections 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 --2- 172.21.15.100:0/1374723022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 -- 172.21.15.100:0/1374723022 >> 172.21.15.100:0/1374723022 conn(0x7f3f0c0678d0 msgr2=0x7f3f0c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.280+0000 7f3f0b59e700 1 -- 172.21.15.100:0/1374723022 shutdown_connections 2024-08-17T02:22:24.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 -- 172.21.15.100:0/1374723022 wait complete. 2024-08-17T02:22:24.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 Processor -- start 2024-08-17T02:22:24.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 -- start start 2024-08-17T02:22:24.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3f0c13c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:24.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c147390 0x7f3f0c1477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:24.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c13ee90 0x7f3f0c13f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:24.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c13cd70 con 0x7f3ef8004810 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c159e30 con 0x7f3f0c147390 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c147d20 con 0x7f3f0c13ee90 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3f0c13c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3f0c13c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37290/0 (socket says 172.21.15.100:37290) 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 -- 172.21.15.100:0/2373144445 learned_addr learned my addr 172.21.15.100:0/2373144445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f03fff700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c147390 0x7f3f0c1477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0ad9d700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c13ee90 0x7f3f0c13f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 -- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c13ee90 msgr2=0x7f3f0c13f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c13ee90 0x7f3f0c13f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 -- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c147390 msgr2=0x7f3f0c1477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c147390 0x7f3f0c1477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 -- 172.21.15.100:0/2373144445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c159ae0 con 0x7f3ef8004810 2024-08-17T02:22:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0ad9d700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c13ee90 0x7f3f0c13f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0a59c700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3f0c13c830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00f940 tx=0x7f3efc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f027fc700 1 -- 172.21.15.100:0/2373144445 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc014670 con 0x7f3ef8004810 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c13fbc0 con 0x7f3ef8004810 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f027fc700 1 -- 172.21.15.100:0/2373144445 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc0147d0 con 0x7f3ef8004810 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f027fc700 1 -- 172.21.15.100:0/2373144445 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc014b90 con 0x7f3ef8004810 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.284+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c140180 con 0x7f3ef8004810 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.288+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c159c70 con 0x7f3ef8004810 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.288+0000 7f3f027fc700 1 -- 172.21.15.100:0/2373144445 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3efc015410 con 0x7f3ef8004810 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.288+0000 7f3f027fc700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3eec060890 0x7f3eec062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.288+0000 7f3f027fc700 1 -- 172.21.15.100:0/2373144445 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3efc07d5a0 con 0x7f3ef8004810 2024-08-17T02:22:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.288+0000 7f3f03fff700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3eec060890 0x7f3eec062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.288+0000 7f3f03fff700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3eec060890 0x7f3eec062d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4001ed0 tx=0x7f3ef4015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:24.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.296+0000 7f3f027fc700 1 -- 172.21.15.100:0/2373144445 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3f0c159c70 con 0x7f3ef8004810 2024-08-17T02:22:24.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.500+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3f0c159c70 con 0x7f3ef8004810 2024-08-17T02:22:24.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.500+0000 7f3f027fc700 1 -- 172.21.15.100:0/2373144445 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3f0c159c70 con 0x7f3ef8004810 2024-08-17T02:22:24.503 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:22:24.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3eec060890 msgr2=0x7f3eec062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:24.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3eec060890 0x7f3eec062d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4001ed0 tx=0x7f3ef4015000 comp rx=0 tx=0).stop 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 msgr2=0x7f3f0c13c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3f0c13c830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00f940 tx=0x7f3efc008040 comp rx=0 tx=0).stop 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 shutdown_connections 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3eec060890 0x7f3eec062d40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c13ee90 0x7f3f0c13f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c147390 0x7f3f0c1477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 --2- 172.21.15.100:0/2373144445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ef8004810 0x7f3f0c13c830 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 >> 172.21.15.100:0/2373144445 conn(0x7f3f0c0678d0 msgr2=0x7f3f0c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 shutdown_connections 2024-08-17T02:22:24.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:24.504+0000 7f3f0b59e700 1 -- 172.21.15.100:0/2373144445 wait complete. 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:22:24.517 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-08-17T02:22:24.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.518 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-08-17T02:22:24.523 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:24.523 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:24.523 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:24.523 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.523 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:24.523 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.523 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42615sec preferred_lft 42615sec 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-17T02:22:24.524 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:7a:f5:6f:c6 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether be:02:20:d6:7f:e2 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::784b:ceff:fe95:7c38/64 scope link 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether be:02:20:d6:7f:e2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::bc02:20ff:fed6:7fe2/64 scope link 2024-08-17T02:22:24.525 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.526 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:22:24.583 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-08-17T02:22:24.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.585 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-08-17T02:22:24.594 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-08-17T02:22:24.594 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:22:24.594 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-17T02:22:24.594 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:22:24.594 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-17T02:22:24.594 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-17T02:22:24.594 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-17T02:22:24.594 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:22:24.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.595 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:24.595 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:24.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:22:24.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:22:24.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:22:24.595 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:24.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.635 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-08-17T02:22:24.683 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-17T02:22:24.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.684 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-08-17T02:22:24.691 INFO:teuthology.orchestra.run.smithi100.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-08-17T02:22:24.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.691 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:24.692 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:24.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-17T02:22:24.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-08-17T02:22:24.692 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:24.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-17T02:22:24.781 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:c3:80:64:bd:bf brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-17T02:22:24.782 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.783 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc3:80ff:fe64:bdbf/64 scope link 2024-08-17T02:22:24.783 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-17T02:22:24.794 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:c3:80:64:bd:bf brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc3:80ff:fe64:bdbf/64 scope link 2024-08-17T02:22:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:24.797 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-08-17T02:22:24.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.797 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:24.797 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:24.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-08-17T02:22:24.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-17T02:22:24.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-08-17T02:22:24.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-08-17T02:22:24.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-08-17T02:22:24.798 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:24.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:24.856 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:24.856 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:24.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.1 up 2024-08-17T02:22:24.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.1 master ceph-brx 2024-08-17T02:22:24.856 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:24.883 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-08-17T02:22:24.883 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:24.883 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-08-17T02:22:24.888 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-08-17T02:22:24.889 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:24.889 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-08-17T02:22:24.895 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:24.957 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:24.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:24.971 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:24.972 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:24.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:24.973 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:24.978 INFO:teuthology.orchestra.run.smithi100.stdout:52 2024-08-17T02:22:24.978 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-08-17T02:22:24.978 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-08-17T02:22:25.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:25.039 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:25.039 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:25.048 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:25.049 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:25.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:25.049 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:25.055 INFO:teuthology.orchestra.run.smithi100.stdout:52 2024-08-17T02:22:25.106 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi100.stderr:2024-08-17T02:22:25.104+0000 7fb617b5d380 -1 init, newargv = 0x55940bd6a880 newargc=15 2024-08-17T02:22:25.106 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi100.stderr:ceph-fuse[30763]: starting ceph client 2024-08-17T02:22:25.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi100.stderr:ceph-fuse[30763]: starting fuse 2024-08-17T02:22:26.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:26.175 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:26.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:26.230 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:26.231 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:26.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:26.232 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:26.239 INFO:teuthology.orchestra.run.smithi100.stdout:52 2024-08-17T02:22:26.239 INFO:teuthology.orchestra.run.smithi100.stdout:53 2024-08-17T02:22:26.239 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-08-17T02:22:26.239 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-08-17T02:22:26.240 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.2")) 2024-08-17T02:22:26.241 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-08-17T02:22:26.323 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.2.30763.asok 2024-08-17T02:22:26.509 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.30763.asok 2024-08-17T02:22:26.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:26.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.30763.asok status 2024-08-17T02:22:26.607 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:22:26.607 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-08-17T02:22:26.607 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "2577e8139c6b8786f9d6d6ea21b8da9244b9260e", 2024-08-17T02:22:26.607 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 17.2.7-1375-g2577e813 (2577e8139c6b8786f9d6d6ea21b8da9244b9260e) quincy (stable)", 2024-08-17T02:22:26.607 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "2", 2024-08-17T02:22:26.607 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-08-17T02:22:26.607 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-08-17T02:22:26.607 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "30763", 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4550, 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4550 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-08-17T02:22:26.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1663337986 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1663337986 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4550 172.21.15.100:0/1663337986", 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "172.21.15.100:0/1663337986", 2024-08-17T02:22:26.609 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-08-17T02:22:26.610 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 6, 2024-08-17T02:22:26.610 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 22, 2024-08-17T02:22:26.610 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-08-17T02:22:26.610 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-08-17T02:22:26.610 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-08-17T02:22:26.610 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:22:26.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:22:26.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 Processor -- start 2024-08-17T02:22:26.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- start start 2024-08-17T02:22:26.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700a8f50 0x7f71700a9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:26.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71700af030 0x7f71701598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:26.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71700b00d0 con 0x7f71700b0560 2024-08-17T02:22:26.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7170159de0 con 0x7f71700af030 2024-08-17T02:22:26.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700af4c0 con 0x7f71700a8f50 2024-08-17T02:22:26.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716effd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71700af030 0x7f71701598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:26.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716f7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700a8f50 0x7f71700a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:26.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716ffff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f71700b0560 0x7f71700b0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:60740/0 (socket says 172.21.15.100:60740) 2024-08-17T02:22:26.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716ffff700 1 -- 172.21.15.100:0/2303589586 learned_addr learned my addr 172.21.15.100:0/2303589586 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:26.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716effd700 1 -- 172.21.15.100:0/2303589586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700a8f50 msgr2=0x7f71700a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:26.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716effd700 1 --2- 172.21.15.100:0/2303589586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700a8f50 0x7f71700a9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716effd700 1 -- 172.21.15.100:0/2303589586 >> v1:172.21.15.123:6789/0 conn(0x7f71700b0560 legacy=0x7f71700b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:26.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716effd700 1 -- 172.21.15.100:0/2303589586 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700af7a0 con 0x7f71700af030 2024-08-17T02:22:26.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716effd700 1 --2- 172.21.15.100:0/2303589586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 0x7f71701598a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f71600098e0 tx=0x7f71600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a66ae16ecc5dd72 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:26.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716e7fc700 1 -- 172.21.15.100:0/2303589586 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000b640 con 0x7f71700af030 2024-08-17T02:22:26.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716e7fc700 1 -- 172.21.15.100:0/2303589586 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716000b7a0 con 0x7f71700af030 2024-08-17T02:22:26.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f716e7fc700 1 -- 172.21.15.100:0/2303589586 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000bb40 con 0x7f71700af030 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- 172.21.15.100:0/2303589586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 msgr2=0x7f71701598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 --2- 172.21.15.100:0/2303589586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 0x7f71701598a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f71600098e0 tx=0x7f71600091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- 172.21.15.100:0/2303589586 shutdown_connections 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 --2- 172.21.15.100:0/2303589586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 0x7f71701598a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 --2- 172.21.15.100:0/2303589586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700a8f50 0x7f71700a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- 172.21.15.100:0/2303589586 >> 172.21.15.100:0/2303589586 conn(0x7f7170066e00 msgr2=0x7f7170067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- 172.21.15.100:0/2303589586 shutdown_connections 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.724+0000 7f7175fe1700 1 -- 172.21.15.100:0/2303589586 wait complete. 2024-08-17T02:22:26.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f7175fe1700 1 Processor -- start 2024-08-17T02:22:26.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f7175fe1700 1 -- start start 2024-08-17T02:22:26.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f7175fe1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700a8f50 0x7f717014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:26.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f7175fe1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 0x7f71700ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:26.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f7175fe1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700b0560 0x7f71700c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:26.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f7175fe1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700ac650 con 0x7f71700a8f50 2024-08-17T02:22:26.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f7175fe1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700acae0 con 0x7f71700af030 2024-08-17T02:22:26.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f7175fe1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700aba40 con 0x7f71700b0560 2024-08-17T02:22:26.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700a8f50 0x7f717014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:26.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700a8f50 0x7f717014da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37306/0 (socket says 172.21.15.100:37306) 2024-08-17T02:22:26.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 -- 172.21.15.100:0/3867663736 learned_addr learned my addr 172.21.15.100:0/3867663736 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716effd700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 0x7f71700ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716ffff700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700b0560 0x7f71700c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 -- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700b0560 msgr2=0x7f71700c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700b0560 0x7f71700c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 -- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 msgr2=0x7f71700ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 0x7f71700ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 -- 172.21.15.100:0/3867663736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009420 con 0x7f71700a8f50 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716effd700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 0x7f71700ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.728+0000 7f716f7fe700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700a8f50 0x7f717014da00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f716400ec80 tx=0x7f71640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f71577fe700 1 -- 172.21.15.100:0/3867663736 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400c850 con 0x7f71700a8f50 2024-08-17T02:22:26.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f71577fe700 1 -- 172.21.15.100:0/3867663736 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716400c9b0 con 0x7f71700a8f50 2024-08-17T02:22:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71700abd20 con 0x7f71700a8f50 2024-08-17T02:22:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f71577fe700 1 -- 172.21.15.100:0/3867663736 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400cd50 con 0x7f71700a8f50 2024-08-17T02:22:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71700ac2e0 con 0x7f71700a8f50 2024-08-17T02:22:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71700645c0 con 0x7f71700a8f50 2024-08-17T02:22:26.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f71577fe700 1 -- 172.21.15.100:0/3867663736 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7164013680 con 0x7f71700a8f50 2024-08-17T02:22:26.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f71577fe700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f715c060860 0x7f715c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:26.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f71577fe700 1 -- 172.21.15.100:0/3867663736 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f716407ce80 con 0x7f71700a8f50 2024-08-17T02:22:26.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.732+0000 7f716effd700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f715c060860 0x7f715c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:26.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.736+0000 7f716effd700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f715c060860 0x7f715c062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71600093f0 tx=0x7f7160017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:26.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.740+0000 7f71577fe700 1 -- 172.21.15.100:0/3867663736 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f716404d230 con 0x7f71700a8f50 2024-08-17T02:22:26.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.944+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f717006f360 con 0x7f71700a8f50 2024-08-17T02:22:26.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f71577fe700 1 -- 172.21.15.100:0/3867663736 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7164050850 con 0x7f71700a8f50 2024-08-17T02:22:26.949 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:22:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f715c060860 msgr2=0x7f715c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f715c060860 0x7f715c062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71600093f0 tx=0x7f7160017000 comp rx=0 tx=0).stop 2024-08-17T02:22:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700a8f50 msgr2=0x7f717014da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700a8f50 0x7f717014da00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f716400ec80 tx=0x7f71640091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 shutdown_connections 2024-08-17T02:22:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f715c060860 0x7f715c062d10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71700b0560 0x7f71700c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71700af030 0x7f71700ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 --2- 172.21.15.100:0/3867663736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700a8f50 0x7f717014da00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 >> 172.21.15.100:0/3867663736 conn(0x7f7170066e00 msgr2=0x7f71700671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 shutdown_connections 2024-08-17T02:22:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:26.948+0000 7f7175fe1700 1 -- 172.21.15.100:0/3867663736 wait complete. 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:22:26.963 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:22:26.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:26.964 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:26.972 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42610sec preferred_lft 42610sec 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 02:42:7f:c5:f2:c5 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:26.973 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-17T02:22:26.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:26.974 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:26.974 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:26.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-08-17T02:22:26.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-08-17T02:22:26.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-08-17T02:22:26.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-17T02:22:26.974 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:27.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:27.060 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:22:27.070 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-08-17T02:22:27.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:27.072 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-08-17T02:22:27.091 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-08-17T02:22:27.091 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:22:27.091 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-17T02:22:27.091 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:22:27.091 INFO:teuthology.orchestra.run.smithi123.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-17T02:22:27.091 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-17T02:22:27.091 INFO:teuthology.orchestra.run.smithi123.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-17T02:22:27.091 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:22:27.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:27.092 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:27.092 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:27.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:22:27.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:22:27.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:22:27.092 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:27.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:27.128 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-08-17T02:22:27.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:27.178 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-08-17T02:22:27.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:27.185 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:27.185 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:27.185 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-17T02:22:27.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-08-17T02:22:27.186 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:27.220 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-08-17T02:22:27.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:27.220 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:27.220 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:27.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-08-17T02:22:27.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-17T02:22:27.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-08-17T02:22:27.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-08-17T02:22:27.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-08-17T02:22:27.222 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:27.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:27.296 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:27.297 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:27.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-08-17T02:22:27.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-08-17T02:22:27.297 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:27.322 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-08-17T02:22:27.326 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:27.326 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-17T02:22:27.332 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:22:27.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:27.332 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-17T02:22:27.338 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:22:27.352 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:27.352 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:27.409 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:27.409 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:27.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:27.410 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:22:27.417 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-17T02:22:27.418 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-08-17T02:22:27.421 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:22:27.435 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:27.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:27.445 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:27.446 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:27.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:27.447 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:22:27.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:2024-08-17T02:22:27.501+0000 7f25a2d4c380 -1 init, newargv = 0x55cf975dea60 newargc=15 2024-08-17T02:22:27.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[29330]: starting ceph client 2024-08-17T02:22:27.514 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[29330]: starting fuse 2024-08-17T02:22:28.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:22:28.618 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:28.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:28.632 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:28.633 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:28.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:28.634 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:22:28.681 INFO:teuthology.orchestra.run.smithi123.stdout:52 2024-08-17T02:22:28.681 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-17T02:22:28.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-08-17T02:22:28.682 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.1")) 2024-08-17T02:22:28.683 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-08-17T02:22:28.767 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.1.29330.asok 2024-08-17T02:22:28.953 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29330.asok 2024-08-17T02:22:28.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:28.954 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29330.asok status 2024-08-17T02:22:29.050 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-08-17T02:22:29.050 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-08-17T02:22:29.050 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "2577e8139c6b8786f9d6d6ea21b8da9244b9260e", 2024-08-17T02:22:29.050 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.7-1375-g2577e813 (2577e8139c6b8786f9d6d6ea21b8da9244b9260e) quincy (stable)", 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "1", 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "29330", 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4562, 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-08-17T02:22:29.051 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4562 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1659207354 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1659207354 2024-08-17T02:22:29.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4562 192.168.0.1:0/1659207354", 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/1659207354", 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 6, 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 22, 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-08-17T02:22:29.053 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-08-17T02:22:29.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:22:29.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.176+0000 7f2217de8700 1 Processor -- start 2024-08-17T02:22:29.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.176+0000 7f2217de8700 1 -- start start 2024-08-17T02:22:29.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.176+0000 7f2217de8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210157060 0x7f221006e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:29.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.176+0000 7f2217de8700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22100c36f0 con 0x7f221006a310 2024-08-17T02:22:29.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.176+0000 7f2217de8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22100caec0 con 0x7f2210068db0 2024-08-17T02:22:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.176+0000 7f2217de8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100ca2b0 con 0x7f2210157060 2024-08-17T02:22:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2215383700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f221006a310 0x7f22100686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58774/0 (socket says 172.21.15.100:58774) 2024-08-17T02:22:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2215383700 1 -- 172.21.15.100:0/3601826830 learned_addr learned my addr 172.21.15.100:0/3601826830 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2215b84700 1 --2- 172.21.15.100:0/3601826830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210157060 0x7f221006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2214b82700 1 -- 172.21.15.100:0/3601826830 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651604383 0 0) 0x7f22100c36f0 con 0x7f221006a310 2024-08-17T02:22:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2214b82700 1 -- 172.21.15.100:0/3601826830 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21fc003570 con 0x7f221006a310 2024-08-17T02:22:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2214b82700 1 -- 172.21.15.100:0/3601826830 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762900659 0 0) 0x7f22100caec0 con 0x7f2210068db0 2024-08-17T02:22:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2214b82700 1 -- 172.21.15.100:0/3601826830 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22100c36f0 con 0x7f2210068db0 2024-08-17T02:22:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2215b84700 1 -- 172.21.15.100:0/3601826830 >> v1:172.21.15.123:6789/0 conn(0x7f2210068db0 legacy=0x7f2210160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2215b84700 1 -- 172.21.15.100:0/3601826830 >> v1:172.21.15.100:6789/0 conn(0x7f221006a310 legacy=0x7f22100686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2214b82700 1 -- 172.21.15.100:0/3601826830 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 634058401 0 0) 0x7f21fc003570 con 0x7f221006a310 2024-08-17T02:22:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2215b84700 1 -- 172.21.15.100:0/3601826830 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100ca590 con 0x7f2210157060 2024-08-17T02:22:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2215b84700 1 --2- 172.21.15.100:0/3601826830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210157060 0x7f221006e130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f220c009910 tx=0x7f220c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ee163180318f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2214b82700 1 -- 172.21.15.100:0/3601826830 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200001470 con 0x7f2210157060 2024-08-17T02:22:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2214b82700 1 -- 172.21.15.100:0/3601826830 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c011420 con 0x7f2210157060 2024-08-17T02:22:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2214b82700 1 -- 172.21.15.100:0/3601826830 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c0117c0 con 0x7f2210157060 2024-08-17T02:22:29.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2217de8700 1 -- 172.21.15.100:0/3601826830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210157060 msgr2=0x7f221006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:29.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2217de8700 1 --2- 172.21.15.100:0/3601826830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210157060 0x7f221006e130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f220c009910 tx=0x7f220c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:29.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2217de8700 1 -- 172.21.15.100:0/3601826830 shutdown_connections 2024-08-17T02:22:29.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2217de8700 1 --2- 172.21.15.100:0/3601826830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210157060 0x7f221006e130 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:29.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2217de8700 1 -- 172.21.15.100:0/3601826830 >> 172.21.15.100:0/3601826830 conn(0x7f22100c9c20 msgr2=0x7f221006d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:29.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2217de8700 1 -- 172.21.15.100:0/3601826830 shutdown_connections 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.180+0000 7f2217de8700 1 -- 172.21.15.100:0/3601826830 wait complete. 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 Processor -- start 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 -- start start 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210068db0 0x7f22100b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221006a310 0x7f221015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2210157060 0x7f2210155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2210158600 con 0x7f221006a310 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100b18d0 con 0x7f2210157060 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22101579f0 con 0x7f2210068db0 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221006a310 0x7f221015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:29.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221006a310 0x7f221015bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37314/0 (socket says 172.21.15.100:37314) 2024-08-17T02:22:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 -- 172.21.15.100:0/83290885 learned_addr learned my addr 172.21.15.100:0/83290885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2216385700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2210157060 0x7f2210155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215b84700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210068db0 0x7f22100b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 -- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210068db0 msgr2=0x7f22100b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210068db0 0x7f22100b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 -- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2210157060 msgr2=0x7f2210155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2210157060 0x7f2210155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 -- 172.21.15.100:0/83290885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c009420 con 0x7f221006a310 2024-08-17T02:22:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215b84700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210068db0 0x7f22100b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2215383700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221006a310 0x7f221015bf50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f220000cba0 tx=0x7f2200001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2205ffb700 1 -- 172.21.15.100:0/83290885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200007e90 con 0x7f221006a310 2024-08-17T02:22:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2205ffb700 1 -- 172.21.15.100:0/83290885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200007360 con 0x7f221006a310 2024-08-17T02:22:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2210157cd0 con 0x7f221006a310 2024-08-17T02:22:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2205ffb700 1 -- 172.21.15.100:0/83290885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200004830 con 0x7f221006a310 2024-08-17T02:22:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.184+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2210158290 con 0x7f221006a310 2024-08-17T02:22:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.188+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22101587b0 con 0x7f221006a310 2024-08-17T02:22:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.188+0000 7f2205ffb700 1 -- 172.21.15.100:0/83290885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2200004ba0 con 0x7f221006a310 2024-08-17T02:22:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.188+0000 7f2205ffb700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f21f4060860 0x7f21f4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.188+0000 7f2205ffb700 1 -- 172.21.15.100:0/83290885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f220007c0e0 con 0x7f221006a310 2024-08-17T02:22:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.188+0000 7f2215b84700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f21f4060860 0x7f21f4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:29.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.188+0000 7f2215b84700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f21f4060860 0x7f21f4062d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f220c0030c0 tx=0x7f220c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:29.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.196+0000 7f2205ffb700 1 -- 172.21.15.100:0/83290885 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f22101587b0 con 0x7f221006a310 2024-08-17T02:22:29.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22101587b0 con 0x7f221006a310 2024-08-17T02:22:29.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2205ffb700 1 -- 172.21.15.100:0/83290885 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f22101587b0 con 0x7f221006a310 2024-08-17T02:22:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:22:29.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f21f4060860 msgr2=0x7f21f4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:29.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f21f4060860 0x7f21f4062d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f220c0030c0 tx=0x7f220c016000 comp rx=0 tx=0).stop 2024-08-17T02:22:29.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221006a310 msgr2=0x7f221015bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:29.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221006a310 0x7f221015bf50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f220000cba0 tx=0x7f2200001b50 comp rx=0 tx=0).stop 2024-08-17T02:22:29.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 shutdown_connections 2024-08-17T02:22:29.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f21f4060860 0x7f21f4062d10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:29.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2210157060 0x7f2210155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:29.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221006a310 0x7f221015bf50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:29.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 --2- 172.21.15.100:0/83290885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2210068db0 0x7f22100b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:29.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 >> 172.21.15.100:0/83290885 conn(0x7f22100c9c20 msgr2=0x7f2210067690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:29.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 shutdown_connections 2024-08-17T02:22:29.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:29.396+0000 7f2217de8700 1 -- 172.21.15.100:0/83290885 wait complete. 2024-08-17T02:22:29.412 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:22:29.412 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:22:29.413 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-08-17T02:22:29.413 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-08-17T02:22:29.413 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-08-17T02:22:29.413 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:22:29.413 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:22:29.414 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:22:29.414 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-08-17T02:22:29.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.414 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-08-17T02:22:29.424 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:29.424 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:29.424 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:29.424 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.424 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:29.425 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.425 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:29.425 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:29.425 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:29.425 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:29.425 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:29.426 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:29.426 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:22:29.426 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:29.426 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-17T02:22:29.426 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42608sec preferred_lft 42608sec 2024-08-17T02:22:29.426 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-08-17T02:22:29.427 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.427 INFO:teuthology.orchestra.run.smithi123.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-17T02:22:29.427 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 02:42:7f:c5:f2:c5 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:29.427 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-17T02:22:29.427 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.428 INFO:teuthology.orchestra.run.smithi123.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:22:29.428 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 12:60:a8:56:33:42 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:29.428 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-17T02:22:29.428 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.428 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::94ae:8dff:fe1e:f582/64 scope link 2024-08-17T02:22:29.428 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.429 INFO:teuthology.orchestra.run.smithi123.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:22:29.429 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 12:60:a8:56:33:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-17T02:22:29.429 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::1060:a8ff:fe56:3342/64 scope link 2024-08-17T02:22:29.429 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.430 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:22:29.446 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-08-17T02:22:29.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.448 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-08-17T02:22:29.457 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-08-17T02:22:29.457 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:22:29.458 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-17T02:22:29.458 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:22:29.458 INFO:teuthology.orchestra.run.smithi123.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-17T02:22:29.458 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-17T02:22:29.459 INFO:teuthology.orchestra.run.smithi123.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-17T02:22:29.459 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:22:29.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.459 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:29.459 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:29.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:22:29.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:22:29.460 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:22:29.460 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:29.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.545 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-08-17T02:22:29.552 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-17T02:22:29.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.553 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-08-17T02:22:29.560 INFO:teuthology.orchestra.run.smithi123.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-08-17T02:22:29.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.561 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:29.561 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:29.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-17T02:22:29.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-08-17T02:22:29.561 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:29.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.589 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-17T02:22:29.646 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:29.646 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:29.646 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:29.646 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.646 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:29.646 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.646 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:22:29.646 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 32:f4:7d:f8:46:ce brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-17T02:22:29.647 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-17T02:22:29.647 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.647 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::30f4:7dff:fef8:46ce/64 scope link 2024-08-17T02:22:29.647 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-17T02:22:29.664 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 32:f4:7d:f8:46:ce brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-17T02:22:29.665 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.666 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::30f4:7dff:fef8:46ce/64 scope link 2024-08-17T02:22:29.666 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:29.667 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-08-17T02:22:29.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.667 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:29.667 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:29.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-08-17T02:22:29.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-17T02:22:29.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-08-17T02:22:29.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-08-17T02:22:29.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-08-17T02:22:29.668 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:29.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:29.729 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:22:29.729 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:22:29.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.1 up 2024-08-17T02:22:29.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.1 master ceph-brx 2024-08-17T02:22:29.729 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:22:29.755 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-08-17T02:22:29.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:29.755 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-08-17T02:22:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-08-17T02:22:29.762 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:29.762 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-08-17T02:22:29.768 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:22:29.828 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:29.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:29.843 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:29.844 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:29.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:29.844 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:22:29.849 INFO:teuthology.orchestra.run.smithi123.stdout:52 2024-08-17T02:22:29.849 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-08-17T02:22:29.850 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-08-17T02:22:29.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:22:29.867 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:29.867 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:29.925 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:29.926 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:29.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:29.926 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:22:29.931 INFO:teuthology.orchestra.run.smithi123.stdout:52 2024-08-17T02:22:29.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:2024-08-17T02:22:29.933+0000 7f160276a380 -1 init, newargv = 0x55bc21d20b10 newargc=15 2024-08-17T02:22:29.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:ceph-fuse[29442]: starting ceph client 2024-08-17T02:22:29.947 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:ceph-fuse[29442]: starting fuse 2024-08-17T02:22:31.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:22:31.052 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:31.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:31.110 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:31.111 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:31.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.112 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:22:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:52 2024-08-17T02:22:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:53 2024-08-17T02:22:31.120 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-08-17T02:22:31.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-17T02:22:31.120 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-08-17T02:22:31.120 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-08-17T02:22:31.120 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-08-17T02:22:31.120 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-08-17T02:22:31.120 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:22:31.120 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-08-17T02:22:31.120 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-17T02:22:31.121 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-17T02:22:31.122 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-08-17T02:22:31.122 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-08-17T02:22:31.122 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-08-17T02:22:31.122 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-17T02:22:31.122 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:22:31.122 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.3")) 2024-08-17T02:22:31.122 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-08-17T02:22:31.203 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.3.29442.asok 2024-08-17T02:22:31.387 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.29442.asok 2024-08-17T02:22:31.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.29442.asok status 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "2577e8139c6b8786f9d6d6ea21b8da9244b9260e", 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.7-1375-g2577e813 (2577e8139c6b8786f9d6d6ea21b8da9244b9260e) quincy (stable)", 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "3", 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "29442", 2024-08-17T02:22:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4672, 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4672 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2024-08-17T02:22:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1182169265 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1182169265 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4672 192.168.0.2:0/1182169265", 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.2:0/1182169265", 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 6, 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 22, 2024-08-17T02:22:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-08-17T02:22:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-08-17T02:22:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-08-17T02:22:31.536 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-08-17T02:22:31.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.543 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-08-17T02:22:31.543 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:31.551 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-08-17T02:22:31.551 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:31.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:31.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.567 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-08-17T02:22:31.567 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.2 2024-08-17T02:22:31.616 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-08-17T02:22:31.616 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-08-17T02:22:31.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-08-17T02:22:31.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.636 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-08-17T02:22:31.636 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.1 2024-08-17T02:22:31.644 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-08-17T02:22:31.644 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-08-17T02:22:31.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-17T02:22:31.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.705 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-08-17T02:22:31.705 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.3 2024-08-17T02:22:31.713 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-08-17T02:22:31.713 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-08-17T02:22:31.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:31.714 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-08-17T02:22:31.727 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-08-17T02:22:31.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:31.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc12baf700 1 Processor -- start 2024-08-17T02:22:31.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc12baf700 1 -- start start 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc12baf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efc0c154900 0x7efc0c0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc12baf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc12baf700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0c0b1730 con 0x7efc0c0b12d0 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc12baf700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc0c068790 con 0x7efc0c154900 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc12baf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c1583f0 con 0x7efc0c0cb560 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc113ac700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7efc0c0b12d0 0x7efc0c0cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58790/0 (socket says 172.21.15.100:58790) 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc113ac700 1 -- 172.21.15.100:0/1512023922 learned_addr learned my addr 172.21.15.100:0/1512023922 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc123ae700 1 --2- 172.21.15.100:0/1512023922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc11bad700 1 --2- 172.21.15.100:0/1512023922 >> v2:172.21.15.123:3300/0 conn(0x7efc0c154900 0x7efc0c0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:31.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc10bab700 1 -- 172.21.15.100:0/1512023922 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213235070 0 0) 0x7efc08001490 con 0x7efc0c0b12d0 2024-08-17T02:22:31.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc10bab700 1 -- 172.21.15.100:0/1512023922 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf4003570 con 0x7efc0c0b12d0 2024-08-17T02:22:31.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc10bab700 1 -- 172.21.15.100:0/1512023922 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608670295 0 0) 0x7efbf4003570 con 0x7efc0c0b12d0 2024-08-17T02:22:31.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc123ae700 1 -- 172.21.15.100:0/1512023922 >> v2:172.21.15.123:3300/0 conn(0x7efc0c154900 msgr2=0x7efc0c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:31.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc123ae700 1 --2- 172.21.15.100:0/1512023922 >> v2:172.21.15.123:3300/0 conn(0x7efc0c154900 0x7efc0c0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:31.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc123ae700 1 -- 172.21.15.100:0/1512023922 >> v1:172.21.15.100:6789/0 conn(0x7efc0c0b12d0 legacy=0x7efc0c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:31.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.860+0000 7efc123ae700 1 -- 172.21.15.100:0/1512023922 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c0cbaa0 con 0x7efc0c0cb560 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc123ae700 1 --2- 172.21.15.100:0/1512023922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c160ac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc040098e0 tx=0x7efc040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3868af65e004a2a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc10bab700 1 -- 172.21.15.100:0/1512023922 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc080019a0 con 0x7efc0c0cb560 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc10bab700 1 -- 172.21.15.100:0/1512023922 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc04011420 con 0x7efc0c0cb560 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc10bab700 1 -- 172.21.15.100:0/1512023922 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040117c0 con 0x7efc0c0cb560 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 -- 172.21.15.100:0/1512023922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 msgr2=0x7efc0c160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 --2- 172.21.15.100:0/1512023922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c160ac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc040098e0 tx=0x7efc040091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 -- 172.21.15.100:0/1512023922 shutdown_connections 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 --2- 172.21.15.100:0/1512023922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c160ac0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 --2- 172.21.15.100:0/1512023922 >> v2:172.21.15.123:3300/0 conn(0x7efc0c154900 0x7efc0c0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 -- 172.21.15.100:0/1512023922 >> 172.21.15.100:0/1512023922 conn(0x7efc0c0678d0 msgr2=0x7efc0c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 -- 172.21.15.100:0/1512023922 shutdown_connections 2024-08-17T02:22:31.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 -- 172.21.15.100:0/1512023922 wait complete. 2024-08-17T02:22:31.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 Processor -- start 2024-08-17T02:22:31.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.864+0000 7efc12baf700 1 -- start start 2024-08-17T02:22:31.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0c0b12d0 0x7efc0c0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:31.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c06a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:31.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc0c154900 0x7efc0c06a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:31.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c144770 con 0x7efc0c0b12d0 2024-08-17T02:22:31.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c144a50 con 0x7efc0c154900 2024-08-17T02:22:31.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c144d30 con 0x7efc0c0cb560 2024-08-17T02:22:31.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0c0b12d0 0x7efc0c0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:31.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0c0b12d0 0x7efc0c0c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37316/0 (socket says 172.21.15.100:37316) 2024-08-17T02:22:31.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 -- 172.21.15.100:0/4281658196 learned_addr learned my addr 172.21.15.100:0/4281658196 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:31.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc113ac700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:31.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc123ae700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc0c154900 0x7efc0c06a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:31.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 -- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 msgr2=0x7efc0c06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:31.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c06a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:31.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 -- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc0c154900 msgr2=0x7efc0c06a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:31.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc0c154900 0x7efc0c06a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 -- 172.21.15.100:0/4281658196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc04009420 con 0x7efc0c0b12d0 2024-08-17T02:22:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc11bad700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0c0b12d0 0x7efc0c0c8080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00e930 tx=0x7efbfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc01ffb700 1 -- 172.21.15.100:0/4281658196 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc013680 con 0x7efc0c0b12d0 2024-08-17T02:22:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc01ffb700 1 -- 172.21.15.100:0/4281658196 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0137e0 con 0x7efc0c0b12d0 2024-08-17T02:22:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc01ffb700 1 -- 172.21.15.100:0/4281658196 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc013b80 con 0x7efc0c0b12d0 2024-08-17T02:22:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0c145010 con 0x7efc0c0b12d0 2024-08-17T02:22:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0c0c85c0 con 0x7efc0c0b12d0 2024-08-17T02:22:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.868+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0c0643a0 con 0x7efc0c0b12d0 2024-08-17T02:22:31.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.872+0000 7efc01ffb700 1 -- 172.21.15.100:0/4281658196 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efbfc00c3e0 con 0x7efc0c0b12d0 2024-08-17T02:22:31.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.872+0000 7efc01ffb700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efbf8060860 0x7efbf8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:31.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.872+0000 7efc01ffb700 1 -- 172.21.15.100:0/4281658196 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efbfc07cd00 con 0x7efc0c0b12d0 2024-08-17T02:22:31.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.872+0000 7efc113ac700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efbf8060860 0x7efbf8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:31.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.872+0000 7efc113ac700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efbf8060860 0x7efbf8062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc08001b50 tx=0x7efc08001760 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:31.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:31.876+0000 7efc01ffb700 1 -- 172.21.15.100:0/4281658196 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7efbfc04d0b0 con 0x7efc0c0b12d0 2024-08-17T02:22:32.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.048+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc0c06f5f0 con 0x7efc0c0b12d0 2024-08-17T02:22:32.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.048+0000 7efc01ffb700 1 -- 172.21.15.100:0/4281658196 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7efbfc0506d0 con 0x7efc0c0b12d0 2024-08-17T02:22:32.052 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:32.052 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:32.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efbf8060860 msgr2=0x7efbf8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efbf8060860 0x7efbf8062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc08001b50 tx=0x7efc08001760 comp rx=0 tx=0).stop 2024-08-17T02:22:32.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0c0b12d0 msgr2=0x7efc0c0c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0c0b12d0 0x7efc0c0c8080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00e930 tx=0x7efbfc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 shutdown_connections 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efbf8060860 0x7efbf8062d10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc0c154900 0x7efc0c06a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc0c0cb560 0x7efc0c06a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 --2- 172.21.15.100:0/4281658196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0c0b12d0 0x7efc0c0c8080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 >> 172.21.15.100:0/4281658196 conn(0x7efc0c0678d0 msgr2=0x7efc0c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 shutdown_connections 2024-08-17T02:22:32.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.052+0000 7efc12baf700 1 -- 172.21.15.100:0/4281658196 wait complete. 2024-08-17T02:22:32.068 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:02.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:03.458625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:22:32.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.176+0000 7fdad4924700 1 Processor -- start 2024-08-17T02:22:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.176+0000 7fdad4924700 1 -- start start 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.176+0000 7fdad4924700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad01590b0 0x7fdad0066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.176+0000 7fdad4924700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdad0159480 0x7fdad0067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.176+0000 7fdad4924700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdad00cd2b0 0x7fdad015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.176+0000 7fdad4924700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad00cba00 con 0x7fdad00cd2b0 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.176+0000 7fdad4924700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad00cdc90 con 0x7fdad0159480 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.176+0000 7fdad4924700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad00ce210 con 0x7fdad01590b0 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdad00cd2b0 0x7fdad015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdad00cd2b0 0x7fdad015ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37330/0 (socket says 172.21.15.100:37330) 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 -- 172.21.15.100:0/1037554063 learned_addr learned my addr 172.21.15.100:0/1037554063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdaceffd700 1 --2- 172.21.15.100:0/1037554063 >> v2:172.21.15.123:3300/0 conn(0x7fdad0159480 0x7fdad0067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacf7fe700 1 --2- 172.21.15.100:0/1037554063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad01590b0 0x7fdad0066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 -- 172.21.15.100:0/1037554063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad01590b0 msgr2=0x7fdad0066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 --2- 172.21.15.100:0/1037554063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad01590b0 0x7fdad0066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 -- 172.21.15.100:0/1037554063 >> v2:172.21.15.123:3300/0 conn(0x7fdad0159480 msgr2=0x7fdad0067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 --2- 172.21.15.100:0/1037554063 >> v2:172.21.15.123:3300/0 conn(0x7fdad0159480 0x7fdad0067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 -- 172.21.15.100:0/1037554063 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad00cb440 con 0x7fdad00cd2b0 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdacffff700 1 --2- 172.21.15.100:0/1037554063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad00cd2b0 0x7fdad015ce00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdad006fa10 tx=0x7fdac8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d411d994ef838e17 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdace7fc700 1 -- 172.21.15.100:0/1037554063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac800e070 con 0x7fdad00cd2b0 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdace7fc700 1 -- 172.21.15.100:0/1037554063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac800b970 con 0x7fdad00cd2b0 2024-08-17T02:22:32.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdace7fc700 1 -- 172.21.15.100:0/1037554063 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80092e0 con 0x7fdad00cd2b0 2024-08-17T02:22:32.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 -- 172.21.15.100:0/1037554063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad00cd2b0 msgr2=0x7fdad015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 --2- 172.21.15.100:0/1037554063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad00cd2b0 0x7fdad015ce00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdad006fa10 tx=0x7fdac8009840 comp rx=0 tx=0).stop 2024-08-17T02:22:32.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 -- 172.21.15.100:0/1037554063 shutdown_connections 2024-08-17T02:22:32.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 --2- 172.21.15.100:0/1037554063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad00cd2b0 0x7fdad015ce00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 --2- 172.21.15.100:0/1037554063 >> v2:172.21.15.123:3300/0 conn(0x7fdad0159480 0x7fdad0067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 --2- 172.21.15.100:0/1037554063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad01590b0 0x7fdad0066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 -- 172.21.15.100:0/1037554063 >> 172.21.15.100:0/1037554063 conn(0x7fdad00d0b90 msgr2=0x7fdad00ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:32.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 -- 172.21.15.100:0/1037554063 shutdown_connections 2024-08-17T02:22:32.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.180+0000 7fdad4924700 1 -- 172.21.15.100:0/1037554063 wait complete. 2024-08-17T02:22:32.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 Processor -- start 2024-08-17T02:22:32.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 -- start start 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad00cd2b0 0x7fdad013c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad01590b0 0x7fdad00c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad0159480 0x7fdad0152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0152e20 con 0x7fdad01590b0 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0153100 con 0x7fdad0159480 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad01533e0 con 0x7fdad00cd2b0 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad01590b0 0x7fdad00c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad01590b0 0x7fdad00c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37346/0 (socket says 172.21.15.100:37346) 2024-08-17T02:22:32.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 -- 172.21.15.100:0/1099141370 learned_addr learned my addr 172.21.15.100:0/1099141370 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdacffff700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad0159480 0x7fdad0152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdacf7fe700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad00cd2b0 0x7fdad013c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 -- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad00cd2b0 msgr2=0x7fdad013c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad00cd2b0 0x7fdad013c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 -- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad0159480 msgr2=0x7fdad0152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad0159480 0x7fdad0152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 -- 172.21.15.100:0/1099141370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac8009ae0 con 0x7fdad01590b0 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdaceffd700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad01590b0 0x7fdad00c9990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e930 tx=0x7fdac00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdab77fe700 1 -- 172.21.15.100:0/1099141370 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac000eed0 con 0x7fdad01590b0 2024-08-17T02:22:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdab77fe700 1 -- 172.21.15.100:0/1099141370 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac000c850 con 0x7fdad01590b0 2024-08-17T02:22:32.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdab77fe700 1 -- 172.21.15.100:0/1099141370 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac000cbf0 con 0x7fdad01590b0 2024-08-17T02:22:32.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad01536c0 con 0x7fdad01590b0 2024-08-17T02:22:32.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.184+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad013ce30 con 0x7fdad01590b0 2024-08-17T02:22:32.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.188+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad0064330 con 0x7fdad01590b0 2024-08-17T02:22:32.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.188+0000 7fdab77fe700 1 -- 172.21.15.100:0/1099141370 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdac0013470 con 0x7fdad01590b0 2024-08-17T02:22:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.188+0000 7fdab77fe700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdabc060860 0x7fdabc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.188+0000 7fdab77fe700 1 -- 172.21.15.100:0/1099141370 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdac007ce50 con 0x7fdad01590b0 2024-08-17T02:22:32.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.188+0000 7fdacf7fe700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdabc060860 0x7fdabc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.192+0000 7fdacf7fe700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdabc060860 0x7fdabc062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009d40 tx=0x7fdac4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.200+0000 7fdab77fe700 1 -- 172.21.15.100:0/1099141370 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdac004d200 con 0x7fdad01590b0 2024-08-17T02:22:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.356+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdad0070cf0 con 0x7fdad01590b0 2024-08-17T02:22:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.356+0000 7fdab77fe700 1 -- 172.21.15.100:0/1099141370 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdac0050820 con 0x7fdad01590b0 2024-08-17T02:22:32.361 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdabc060860 msgr2=0x7fdabc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdabc060860 0x7fdabc062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009d40 tx=0x7fdac4008040 comp rx=0 tx=0).stop 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad01590b0 msgr2=0x7fdad00c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad01590b0 0x7fdad00c9990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e930 tx=0x7fdac00091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 shutdown_connections 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdabc060860 0x7fdabc062d10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad0159480 0x7fdad0152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad01590b0 0x7fdad00c9990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 --2- 172.21.15.100:0/1099141370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdad00cd2b0 0x7fdad013c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 >> 172.21.15.100:0/1099141370 conn(0x7fdad00d0b90 msgr2=0x7fdad0063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:32.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 shutdown_connections 2024-08-17T02:22:32.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.360+0000 7fdad4924700 1 -- 172.21.15.100:0/1099141370 wait complete. 2024-08-17T02:22:32.378 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:22:32.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9e127d700 1 Processor -- start 2024-08-17T02:22:32.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9e127d700 1 -- start start 2024-08-17T02:22:32.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9e127d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9dc159300 0x7fa9dc0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9e127d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc067510 0x7fa9dc0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9e127d700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9dc0cba90 con 0x7fa9dc1596d0 2024-08-17T02:22:32.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9e127d700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc0b2b60 con 0x7fa9dc159300 2024-08-17T02:22:32.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9e127d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0cb160 con 0x7fa9dc067510 2024-08-17T02:22:32.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9da7fc700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa9dc1596d0 0x7fa9dc066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58806/0 (socket says 172.21.15.100:58806) 2024-08-17T02:22:32.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9da7fc700 1 -- 172.21.15.100:0/289978940 learned_addr learned my addr 172.21.15.100:0/289978940 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9daffd700 1 --2- 172.21.15.100:0/289978940 >> v2:172.21.15.100:3300/0 conn(0x7fa9dc159300 0x7fa9dc0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.488+0000 7fa9db7fe700 1 --2- 172.21.15.100:0/289978940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc067510 0x7fa9dc0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9d9ffb700 1 -- 172.21.15.100:0/289978940 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914752736 0 0) 0x7fa9dc0cba90 con 0x7fa9dc1596d0 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9d9ffb700 1 -- 172.21.15.100:0/289978940 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c4003570 con 0x7fa9dc1596d0 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9d9ffb700 1 -- 172.21.15.100:0/289978940 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160710980 0 0) 0x7fa9c4003570 con 0x7fa9dc1596d0 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9daffd700 1 -- 172.21.15.100:0/289978940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc067510 msgr2=0x7fa9dc0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9daffd700 1 --2- 172.21.15.100:0/289978940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc067510 0x7fa9dc0cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9daffd700 1 -- 172.21.15.100:0/289978940 >> v1:172.21.15.100:6789/0 conn(0x7fa9dc1596d0 legacy=0x7fa9dc066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9daffd700 1 -- 172.21.15.100:0/289978940 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0cb440 con 0x7fa9dc159300 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9daffd700 1 --2- 172.21.15.100:0/289978940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc0b2230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00098e0 tx=0x7fa9d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42ba648cab7e00b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9d9ffb700 1 -- 172.21.15.100:0/289978940 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc001470 con 0x7fa9dc159300 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9d9ffb700 1 -- 172.21.15.100:0/289978940 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0011420 con 0x7fa9dc159300 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9d9ffb700 1 -- 172.21.15.100:0/289978940 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00117a0 con 0x7fa9dc159300 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9e127d700 1 -- 172.21.15.100:0/289978940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 msgr2=0x7fa9dc0b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9e127d700 1 --2- 172.21.15.100:0/289978940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc0b2230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00098e0 tx=0x7fa9d00091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9e127d700 1 -- 172.21.15.100:0/289978940 shutdown_connections 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9e127d700 1 --2- 172.21.15.100:0/289978940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc067510 0x7fa9dc0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9e127d700 1 --2- 172.21.15.100:0/289978940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc0b2230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9e127d700 1 -- 172.21.15.100:0/289978940 >> 172.21.15.100:0/289978940 conn(0x7fa9dc0682c0 msgr2=0x7fa9dc0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:32.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9e127d700 1 -- 172.21.15.100:0/289978940 shutdown_connections 2024-08-17T02:22:32.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.492+0000 7fa9e127d700 1 -- 172.21.15.100:0/289978940 wait complete. 2024-08-17T02:22:32.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 Processor -- start 2024-08-17T02:22:32.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 -- start start 2024-08-17T02:22:32.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc067510 0x7fa9dc0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc16d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc1596d0 0x7fa9dc1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0b0d10 con 0x7fa9dc159300 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0c5370 con 0x7fa9dc067510 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc16da30 con 0x7fa9dc1596d0 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc16d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc16d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37356/0 (socket says 172.21.15.100:37356) 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 -- 172.21.15.100:0/1359075594 learned_addr learned my addr 172.21.15.100:0/1359075594 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9daffd700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc067510 0x7fa9dc0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9db7fe700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc1596d0 0x7fa9dc1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 -- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc1596d0 msgr2=0x7fa9dc1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc1596d0 0x7fa9dc1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 -- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc067510 msgr2=0x7fa9dc0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc067510 0x7fa9dc0c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 -- 172.21.15.100:0/1359075594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d0009420 con 0x7fa9dc159300 2024-08-17T02:22:32.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9db7fe700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc1596d0 0x7fa9dc1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:32.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9da7fc700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc16d4f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00cbc0 tx=0x7fa9cc001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9c2ffd700 1 -- 172.21.15.100:0/1359075594 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc007150 con 0x7fa9dc159300 2024-08-17T02:22:32.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9c2ffd700 1 -- 172.21.15.100:0/1359075594 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0072d0 con 0x7fa9dc159300 2024-08-17T02:22:32.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc0b0a30 con 0x7fa9dc159300 2024-08-17T02:22:32.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9c2ffd700 1 -- 172.21.15.100:0/1359075594 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc007690 con 0x7fa9dc159300 2024-08-17T02:22:32.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.496+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc0c5000 con 0x7fa9dc159300 2024-08-17T02:22:32.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.500+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc0645c0 con 0x7fa9dc159300 2024-08-17T02:22:32.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.500+0000 7fa9c2ffd700 1 -- 172.21.15.100:0/1359075594 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa9cc004a40 con 0x7fa9dc159300 2024-08-17T02:22:32.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.500+0000 7fa9c2ffd700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa9c8060860 0x7fa9c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.500+0000 7fa9c2ffd700 1 -- 172.21.15.100:0/1359075594 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa9cc07c360 con 0x7fa9dc159300 2024-08-17T02:22:32.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.500+0000 7fa9daffd700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa9c8060860 0x7fa9c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.500+0000 7fa9daffd700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa9c8060860 0x7fa9c8062d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00091e0 tx=0x7fa9d0016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.508+0000 7fa9c2ffd700 1 -- 172.21.15.100:0/1359075594 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa9cc04c830 con 0x7fa9dc159300 2024-08-17T02:22:32.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.672+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9dc0c54b0 con 0x7fa9dc159300 2024-08-17T02:22:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.672+0000 7fa9c2ffd700 1 -- 172.21.15.100:0/1359075594 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9cc018020 con 0x7fa9dc159300 2024-08-17T02:22:32.675 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:22:32.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa9c8060860 msgr2=0x7fa9c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa9c8060860 0x7fa9c8062d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00091e0 tx=0x7fa9d0016000 comp rx=0 tx=0).stop 2024-08-17T02:22:32.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 msgr2=0x7fa9dc16d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc16d4f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00cbc0 tx=0x7fa9cc001a30 comp rx=0 tx=0).stop 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 shutdown_connections 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa9c8060860 0x7fa9c8062d10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9dc1596d0 0x7fa9dc1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc159300 0x7fa9dc16d4f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 --2- 172.21.15.100:0/1359075594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc067510 0x7fa9dc0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 >> 172.21.15.100:0/1359075594 conn(0x7fa9dc0682c0 msgr2=0x7fa9dc068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 shutdown_connections 2024-08-17T02:22:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.676+0000 7fa9e127d700 1 -- 172.21.15.100:0/1359075594 wait complete. 2024-08-17T02:22:32.688 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:22:32.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.796+0000 7fdce759e700 1 Processor -- start 2024-08-17T02:22:32.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.796+0000 7fdce759e700 1 -- start start 2024-08-17T02:22:32.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce806ead0 0x7fdce806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdce80b1830 0x7fdce8155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce80ace30 con 0x7fdce8152bd0 2024-08-17T02:22:32.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce80b1c80 con 0x7fdce80b1830 2024-08-17T02:22:32.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8156dd0 con 0x7fdce806ead0 2024-08-17T02:22:32.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce6d9d700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdce8152bd0 0x7fdce8153010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58822/0 (socket says 172.21.15.100:58822) 2024-08-17T02:22:32.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdcdffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdce80b1830 0x7fdce8155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce6d9d700 1 -- 172.21.15.100:0/3749887194 learned_addr learned my addr 172.21.15.100:0/3749887194 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:32.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce659c700 1 --2- 172.21.15.100:0/3749887194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce806ead0 0x7fdce806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce5d9b700 1 -- 172.21.15.100:0/3749887194 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482961795 0 0) 0x7fdce80ace30 con 0x7fdce8152bd0 2024-08-17T02:22:32.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce5d9b700 1 -- 172.21.15.100:0/3749887194 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd4003570 con 0x7fdce8152bd0 2024-08-17T02:22:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdcdffff700 1 -- 172.21.15.100:0/3749887194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce806ead0 msgr2=0x7fdce806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdcdffff700 1 --2- 172.21.15.100:0/3749887194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce806ead0 0x7fdce806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdcdffff700 1 -- 172.21.15.100:0/3749887194 >> v1:172.21.15.100:6789/0 conn(0x7fdce8152bd0 legacy=0x7fdce8153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdcdffff700 1 -- 172.21.15.100:0/3749887194 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8156450 con 0x7fdce80b1830 2024-08-17T02:22:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdcdffff700 1 --2- 172.21.15.100:0/3749887194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce80b1830 0x7fdce8155c00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00098e0 tx=0x7fdcd00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=170c681782da7696 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce5d9b700 1 -- 172.21.15.100:0/3749887194 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011630 con 0x7fdce80b1830 2024-08-17T02:22:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce5d9b700 1 -- 172.21.15.100:0/3749887194 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0011790 con 0x7fdce80b1830 2024-08-17T02:22:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce5d9b700 1 -- 172.21.15.100:0/3749887194 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011b30 con 0x7fdce80b1830 2024-08-17T02:22:32.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 -- 172.21.15.100:0/3749887194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce80b1830 msgr2=0x7fdce8155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 --2- 172.21.15.100:0/3749887194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce80b1830 0x7fdce8155c00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00098e0 tx=0x7fdcd00091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 -- 172.21.15.100:0/3749887194 shutdown_connections 2024-08-17T02:22:32.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 --2- 172.21.15.100:0/3749887194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce80b1830 0x7fdce8155c00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 --2- 172.21.15.100:0/3749887194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce806ead0 0x7fdce806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 -- 172.21.15.100:0/3749887194 >> 172.21.15.100:0/3749887194 conn(0x7fdce80678a0 msgr2=0x7fdce8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:32.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 -- 172.21.15.100:0/3749887194 shutdown_connections 2024-08-17T02:22:32.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.800+0000 7fdce759e700 1 -- 172.21.15.100:0/3749887194 wait complete. 2024-08-17T02:22:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.804+0000 7fdce759e700 1 Processor -- start 2024-08-17T02:22:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.804+0000 7fdce759e700 1 -- start start 2024-08-17T02:22:32.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce806ead0 0x7fdce813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce80b1830 0x7fdce813b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8152bd0 0x7fdce8148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce759e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce813af40 con 0x7fdce80b1830 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce813a570 con 0x7fdce806ead0 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce813bc30 con 0x7fdce8152bd0 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce80b1830 0x7fdce813b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce80b1830 0x7fdce813b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37364/0 (socket says 172.21.15.100:37364) 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 -- 172.21.15.100:0/2899393275 learned_addr learned my addr 172.21.15.100:0/2899393275 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce6d9d700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8152bd0 0x7fdce8148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce659c700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce806ead0 0x7fdce813aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 -- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8152bd0 msgr2=0x7fdce8148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8152bd0 0x7fdce8148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 -- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce806ead0 msgr2=0x7fdce813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce806ead0 0x7fdce813aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 -- 172.21.15.100:0/2899393275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0009420 con 0x7fdce80b1830 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce659c700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce806ead0 0x7fdce813aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcdffff700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce80b1830 0x7fdce813b370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdce8068b40 tx=0x7fdcd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcde7fc700 1 -- 172.21.15.100:0/2899393275 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011b10 con 0x7fdce80b1830 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcde7fc700 1 -- 172.21.15.100:0/2899393275 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0000cc0 con 0x7fdce80b1830 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce813bf10 con 0x7fdce80b1830 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdcde7fc700 1 -- 172.21.15.100:0/2899393275 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0023600 con 0x7fdce80b1830 2024-08-17T02:22:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.808+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce813a200 con 0x7fdce80b1830 2024-08-17T02:22:32.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.812+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce813a6e0 con 0x7fdce80b1830 2024-08-17T02:22:32.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.812+0000 7fdcde7fc700 1 -- 172.21.15.100:0/2899393275 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdcd002c3b0 con 0x7fdce80b1830 2024-08-17T02:22:32.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.812+0000 7fdcde7fc700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdcc8060860 0x7fdcc8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:32.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.812+0000 7fdcde7fc700 1 -- 172.21.15.100:0/2899393275 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdcd007df10 con 0x7fdce80b1830 2024-08-17T02:22:32.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.812+0000 7fdce659c700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdcc8060860 0x7fdcc8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:32.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.812+0000 7fdce659c700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdcc8060860 0x7fdcc8062d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009ce0 tx=0x7fdcd8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.816+0000 7fdcde7fc700 1 -- 172.21.15.100:0/2899393275 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdcd004e3e0 con 0x7fdce80b1830 2024-08-17T02:22:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.976+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdce8064330 con 0x7fdce80b1830 2024-08-17T02:22:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.976+0000 7fdcde7fc700 1 -- 172.21.15.100:0/2899393275 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcd00150a0 con 0x7fdce80b1830 2024-08-17T02:22:32.981 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:22:32.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdcc8060860 msgr2=0x7fdcc8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdcc8060860 0x7fdcc8062d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009ce0 tx=0x7fdcd8008040 comp rx=0 tx=0).stop 2024-08-17T02:22:32.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce80b1830 msgr2=0x7fdce813b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce80b1830 0x7fdce813b370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdce8068b40 tx=0x7fdcd0017000 comp rx=0 tx=0).stop 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 shutdown_connections 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdcc8060860 0x7fdcc8062d10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8152bd0 0x7fdce8148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce80b1830 0x7fdce813b370 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 --2- 172.21.15.100:0/2899393275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce806ead0 0x7fdce813aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 >> 172.21.15.100:0/2899393275 conn(0x7fdce80678a0 msgr2=0x7fdce8154c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 shutdown_connections 2024-08-17T02:22:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:32.980+0000 7fdce759e700 1 -- 172.21.15.100:0/2899393275 wait complete. 2024-08-17T02:22:32.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:33.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.092+0000 7fce31216700 1 Processor -- start 2024-08-17T02:22:33.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.092+0000 7fce31216700 1 -- start start 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fce2c0b0d20 0x7fce2c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c0b1800 0x7fce2c141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c141a00 con 0x7fce2c06e450 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c06e8a0 con 0x7fce2c0b0d20 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c140b10 con 0x7fce2c0b1800 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce30a15700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c0b1800 0x7fce2c141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fce2c0b0d20 0x7fce2c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2b7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fce2c06e450 0x7fce2c0b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:60766/0 (socket says 172.21.15.100:60766) 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce30a15700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c0b1800 0x7fce2c141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:58640/0 (socket says 172.21.15.100:58640) 2024-08-17T02:22:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2b7fe700 1 -- 172.21.15.100:0/357772390 learned_addr learned my addr 172.21.15.100:0/357772390 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:33.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2bfff700 1 -- 172.21.15.100:0/357772390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c0b1800 msgr2=0x7fce2c141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:33.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2bfff700 1 --2- 172.21.15.100:0/357772390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c0b1800 0x7fce2c141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2bfff700 1 -- 172.21.15.100:0/357772390 >> v1:172.21.15.123:6789/0 conn(0x7fce2c06e450 legacy=0x7fce2c0b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2bfff700 1 -- 172.21.15.100:0/357772390 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c140df0 con 0x7fce2c0b0d20 2024-08-17T02:22:33.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce30a15700 1 --2- 172.21.15.100:0/357772390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c0b1800 0x7fce2c141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:33.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2bfff700 1 --2- 172.21.15.100:0/357772390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 0x7fce2c06df10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0098e0 tx=0x7fce1c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e163bb1baf96e1c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2affd700 1 -- 172.21.15.100:0/357772390 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00b640 con 0x7fce2c0b0d20 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2affd700 1 -- 172.21.15.100:0/357772390 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c00b7a0 con 0x7fce2c0b0d20 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce2affd700 1 -- 172.21.15.100:0/357772390 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00bb40 con 0x7fce2c0b0d20 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 -- 172.21.15.100:0/357772390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 msgr2=0x7fce2c06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 --2- 172.21.15.100:0/357772390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 0x7fce2c06df10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0098e0 tx=0x7fce1c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 -- 172.21.15.100:0/357772390 shutdown_connections 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 --2- 172.21.15.100:0/357772390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c0b1800 0x7fce2c141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 --2- 172.21.15.100:0/357772390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 0x7fce2c06df10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 -- 172.21.15.100:0/357772390 >> 172.21.15.100:0/357772390 conn(0x7fce2c0678d0 msgr2=0x7fce2c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 -- 172.21.15.100:0/357772390 shutdown_connections 2024-08-17T02:22:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.096+0000 7fce31216700 1 -- 172.21.15.100:0/357772390 wait complete. 2024-08-17T02:22:33.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 Processor -- start 2024-08-17T02:22:33.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 -- start start 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c06e450 0x7fce2c16e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 0x7fce2c15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce2c0b1800 0x7fce2c0a8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c16ea70 con 0x7fce2c0b1800 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0a6200 con 0x7fce2c0b0d20 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0a55f0 con 0x7fce2c06e450 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce2c0b1800 0x7fce2c0a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce2c0b1800 0x7fce2c0a8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37378/0 (socket says 172.21.15.100:37378) 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 -- 172.21.15.100:0/419978866 learned_addr learned my addr 172.21.15.100:0/419978866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce2b7fe700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 0x7fce2c15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce2bfff700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c06e450 0x7fce2c16e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 -- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c06e450 msgr2=0x7fce2c16e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c06e450 0x7fce2c16e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 -- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 msgr2=0x7fce2c15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 0x7fce2c15bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 -- 172.21.15.100:0/419978866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24009a20 con 0x7fce2c0b1800 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce30a15700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce2c0b1800 0x7fce2c0a8530 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce24009fc0 tx=0x7fce2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce23fff700 1 -- 172.21.15.100:0/419978866 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400cc80 con 0x7fce2c0b1800 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce23fff700 1 -- 172.21.15.100:0/419978866 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2400f040 con 0x7fce2c0b1800 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c009420 con 0x7fce2c0b1800 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce23fff700 1 -- 172.21.15.100:0/419978866 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24014480 con 0x7fce2c0b1800 2024-08-17T02:22:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.100+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c0a5bb0 con 0x7fce2c0b1800 2024-08-17T02:22:33.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.104+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c0a6390 con 0x7fce2c0b1800 2024-08-17T02:22:33.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.104+0000 7fce23fff700 1 -- 172.21.15.100:0/419978866 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fce24003020 con 0x7fce2c0b1800 2024-08-17T02:22:33.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.104+0000 7fce23fff700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fce0c060860 0x7fce0c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.104+0000 7fce23fff700 1 -- 172.21.15.100:0/419978866 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce2407d5d0 con 0x7fce2c0b1800 2024-08-17T02:22:33.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.104+0000 7fce2bfff700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fce0c060860 0x7fce0c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.104+0000 7fce2bfff700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fce0c060860 0x7fce0c062d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0093f0 tx=0x7fce1c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.108+0000 7fce23fff700 1 -- 172.21.15.100:0/419978866 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fce2404dad0 con 0x7fce2c0b1800 2024-08-17T02:22:33.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.272+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce2c0643a0 con 0x7fce2c0b1800 2024-08-17T02:22:33.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.272+0000 7fce23fff700 1 -- 172.21.15.100:0/419978866 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fce240510f0 con 0x7fce2c0b1800 2024-08-17T02:22:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:33.277 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:33.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fce0c060860 msgr2=0x7fce0c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fce0c060860 0x7fce0c062d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0093f0 tx=0x7fce1c017000 comp rx=0 tx=0).stop 2024-08-17T02:22:33.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce2c0b1800 msgr2=0x7fce2c0a8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce2c0b1800 0x7fce2c0a8530 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce24009fc0 tx=0x7fce2400b040 comp rx=0 tx=0).stop 2024-08-17T02:22:33.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 shutdown_connections 2024-08-17T02:22:33.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fce0c060860 0x7fce0c062d10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce2c0b1800 0x7fce2c0a8530 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2c0b0d20 0x7fce2c15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 --2- 172.21.15.100:0/419978866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce2c06e450 0x7fce2c16e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 >> 172.21.15.100:0/419978866 conn(0x7fce2c0678d0 msgr2=0x7fce2c15eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:33.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 shutdown_connections 2024-08-17T02:22:33.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.276+0000 7fce31216700 1 -- 172.21.15.100:0/419978866 wait complete. 2024-08-17T02:22:33.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:02.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:03.458625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:22:33.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7f59e700 1 Processor -- start 2024-08-17T02:22:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7f59e700 1 -- start start 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 0x7f4d800aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7f59e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d80140f70 con 0x7f4d8006e840 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d80141250 con 0x7f4d800b1600 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800b1a80 con 0x7f4d8006e470 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7dd9b700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4d8006e840 0x7f4d800b0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58830/0 (socket says 172.21.15.100:58830) 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7dd9b700 1 -- 172.21.15.100:0/3635364268 learned_addr learned my addr 172.21.15.100:0/3635364268 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7e59c700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 0x7f4d800aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787922902 0 0) 0x7f4d80140f70 con 0x7f4d8006e840 2024-08-17T02:22:33.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d68003570 con 0x7f4d8006e840 2024-08-17T02:22:33.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 949773820 0 0) 0x7f4d68003570 con 0x7f4d8006e840 2024-08-17T02:22:33.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 msgr2=0x7f4d800aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:33.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7d59a700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 0x7f4d800aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 >> v1:172.21.15.123:6789/0 conn(0x7f4d800b1600 legacy=0x7f4d80141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80141530 con 0x7f4d8006e840 2024-08-17T02:22:33.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7e59c700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 0x7f4d800aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:33.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.400+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f4d74001470 con 0x7f4d8006e840 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 >> v1:172.21.15.100:6789/0 conn(0x7f4d8006e840 legacy=0x7f4d800b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d68003a90 0x7f4d68003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 0x7f4d680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6800bb10 con 0x7f4d68004810 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6800bc50 con 0x7f4d68003a90 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6800bdc0 con 0x7f4d8006e470 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7e59c700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d68003a90 0x7f4d68003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7dd9b700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7ed9d700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 0x7f4d680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7dd9b700 1 -- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 msgr2=0x7f4d680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7dd9b700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8006e470 0x7f4d680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7dd9b700 1 -- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d68003a90 msgr2=0x7f4d68003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7dd9b700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d68003a90 0x7f4d68003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7dd9b700 1 -- 172.21.15.100:0/3635364268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d78009a20 con 0x7f4d68004810 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7dd9b700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d68004c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c2d0 tx=0x7f4d74001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5dab7921d5298f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400cb00 con 0x7f4d68004810 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7400cc60 con 0x7f4d68004810 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7d59a700 1 -- 172.21.15.100:0/3635364268 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74005920 con 0x7f4d68004810 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7f59e700 1 -- 172.21.15.100:0/3635364268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 msgr2=0x7f4d68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7f59e700 1 --2- 172.21.15.100:0/3635364268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d68004c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c2d0 tx=0x7f4d74001df0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7ed9d700 1 -- 172.21.15.100:0/3635364268 reap_dead start 2024-08-17T02:22:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7f59e700 1 -- 172.21.15.100:0/3635364268 shutdown_connections 2024-08-17T02:22:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7f59e700 1 -- 172.21.15.100:0/3635364268 >> 172.21.15.100:0/3635364268 conn(0x7f4d800678a0 msgr2=0x7f4d80067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7f59e700 1 -- 172.21.15.100:0/3635364268 shutdown_connections 2024-08-17T02:22:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7f59e700 1 -- 172.21.15.100:0/3635364268 wait complete. 2024-08-17T02:22:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7f59e700 1 Processor -- start 2024-08-17T02:22:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.404+0000 7f4d7f59e700 1 -- start start 2024-08-17T02:22:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d800ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8015bc30 0x7f4d800a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d800aca30 0x7f4d800a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800a6190 con 0x7f4d68004810 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800a5580 con 0x7f4d800aca30 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800a5860 con 0x7f4d8015bc30 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d800ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d800ac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37394/0 (socket says 172.21.15.100:37394) 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 -- 172.21.15.100:0/2138957792 learned_addr learned my addr 172.21.15.100:0/2138957792 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7ed9d700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d800aca30 0x7f4d800a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7dd9b700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8015bc30 0x7f4d800a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 -- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8015bc30 msgr2=0x7f4d800a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8015bc30 0x7f4d800a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 -- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d800aca30 msgr2=0x7f4d800a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d800aca30 0x7f4d800a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 -- 172.21.15.100:0/2138957792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800a5b40 con 0x7f4d68004810 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7e59c700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d800ac4f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800f2e0 tx=0x7f4d7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d6e7fc700 1 -- 172.21.15.100:0/2138957792 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800e040 con 0x7f4d68004810 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d6e7fc700 1 -- 172.21.15.100:0/2138957792 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7800f040 con 0x7f4d68004810 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d800a5e20 con 0x7f4d68004810 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d6e7fc700 1 -- 172.21.15.100:0/2138957792 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78015680 con 0x7f4d68004810 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d800aa780 con 0x7f4d68004810 2024-08-17T02:22:33.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.408+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d80064330 con 0x7f4d68004810 2024-08-17T02:22:33.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.412+0000 7f4d6e7fc700 1 -- 172.21.15.100:0/2138957792 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4d78015a30 con 0x7f4d68004810 2024-08-17T02:22:33.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.412+0000 7f4d6e7fc700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4d60060860 0x7f4d60062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.412+0000 7f4d6e7fc700 1 -- 172.21.15.100:0/2138957792 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d7807db60 con 0x7f4d68004810 2024-08-17T02:22:33.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.412+0000 7f4d7dd9b700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4d60060860 0x7f4d60062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.412+0000 7f4d7dd9b700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4d60060860 0x7f4d60062d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d74008fd0 tx=0x7f4d74015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.416+0000 7f4d6e7fc700 1 -- 172.21.15.100:0/2138957792 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4d7804dfb0 con 0x7f4d68004810 2024-08-17T02:22:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.580+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d8006ed70 con 0x7f4d68004810 2024-08-17T02:22:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.580+0000 7f4d6e7fc700 1 -- 172.21.15.100:0/2138957792 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d78019020 con 0x7f4d68004810 2024-08-17T02:22:33.584 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4d60060860 msgr2=0x7f4d60062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4d60060860 0x7f4d60062d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d74008fd0 tx=0x7f4d74015000 comp rx=0 tx=0).stop 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 msgr2=0x7f4d800ac4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d800ac4f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800f2e0 tx=0x7f4d7800b040 comp rx=0 tx=0).stop 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 shutdown_connections 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4d60060860 0x7f4d60062d10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d800aca30 0x7f4d800a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d8015bc30 0x7f4d800a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 --2- 172.21.15.100:0/2138957792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d68004810 0x7f4d800ac4f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 >> 172.21.15.100:0/2138957792 conn(0x7f4d800678a0 msgr2=0x7f4d80141d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 shutdown_connections 2024-08-17T02:22:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.584+0000 7f4d7f59e700 1 -- 172.21.15.100:0/2138957792 wait complete. 2024-08-17T02:22:33.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:22:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.708+0000 7f4ecc616700 1 Processor -- start 2024-08-17T02:22:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.708+0000 7f4ecc616700 1 -- start start 2024-08-17T02:22:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.708+0000 7f4ecc616700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4ec4153a30 0x7f4ec415cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.708+0000 7f4ecc616700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4ec415d3e0 0x7f4ec4163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.708+0000 7f4ecc616700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec40c9df0 con 0x7f4ec4153660 2024-08-17T02:22:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.708+0000 7f4ecc616700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec415d800 con 0x7f4ec4153a30 2024-08-17T02:22:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.708+0000 7f4ecc616700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec40cdca0 con 0x7f4ec415d3e0 2024-08-17T02:22:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.708+0000 7f4ec9bb1700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4ec4153a30 0x7f4ec415cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4eca3b2700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4ec4153660 0x7f4ec40cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:60778/0 (socket says 172.21.15.100:60778) 2024-08-17T02:22:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4eca3b2700 1 -- 172.21.15.100:0/3906624212 learned_addr learned my addr 172.21.15.100:0/3906624212 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecabb3700 1 --2- 172.21.15.100:0/3906624212 >> v2:172.21.15.123:3300/0 conn(0x7f4ec415d3e0 0x7f4ec4163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec93b0700 1 -- 172.21.15.100:0/3906624212 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467720246 0 0) 0x7f4ec40c9df0 con 0x7f4ec4153660 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec93b0700 1 -- 172.21.15.100:0/3906624212 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eac003570 con 0x7f4ec4153660 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec9bb1700 1 -- 172.21.15.100:0/3906624212 >> v2:172.21.15.123:3300/0 conn(0x7f4ec415d3e0 msgr2=0x7f4ec4163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec9bb1700 1 --2- 172.21.15.100:0/3906624212 >> v2:172.21.15.123:3300/0 conn(0x7f4ec415d3e0 0x7f4ec4163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec9bb1700 1 -- 172.21.15.100:0/3906624212 >> v1:172.21.15.123:6789/0 conn(0x7f4ec4153660 legacy=0x7f4ec40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec9bb1700 1 -- 172.21.15.100:0/3906624212 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec40c91e0 con 0x7f4ec4153a30 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec9bb1700 1 --2- 172.21.15.100:0/3906624212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4153a30 0x7f4ec415cea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4157470 tx=0x7f4eb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea066ef8f0b02db server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec93b0700 1 -- 172.21.15.100:0/3906624212 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8011630 con 0x7f4ec4153a30 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec93b0700 1 -- 172.21.15.100:0/3906624212 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb80117b0 con 0x7f4ec4153a30 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ec93b0700 1 -- 172.21.15.100:0/3906624212 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8011b70 con 0x7f4ec4153a30 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecc616700 1 -- 172.21.15.100:0/3906624212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4153a30 msgr2=0x7f4ec415cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecc616700 1 --2- 172.21.15.100:0/3906624212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4153a30 0x7f4ec415cea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4157470 tx=0x7f4eb80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecc616700 1 -- 172.21.15.100:0/3906624212 shutdown_connections 2024-08-17T02:22:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecc616700 1 --2- 172.21.15.100:0/3906624212 >> v2:172.21.15.123:3300/0 conn(0x7f4ec415d3e0 0x7f4ec4163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecc616700 1 --2- 172.21.15.100:0/3906624212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4153a30 0x7f4ec415cea0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecc616700 1 -- 172.21.15.100:0/3906624212 >> 172.21.15.100:0/3906624212 conn(0x7f4ec40caad0 msgr2=0x7f4ec40caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecc616700 1 -- 172.21.15.100:0/3906624212 shutdown_connections 2024-08-17T02:22:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.712+0000 7f4ecc616700 1 -- 172.21.15.100:0/3906624212 wait complete. 2024-08-17T02:22:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 Processor -- start 2024-08-17T02:22:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 -- start start 2024-08-17T02:22:33.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4153660 0x7f4ec40b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ec4153a30 0x7f4ec415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec415d3e0 0x7f4ec413a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec413be90 con 0x7f4ec415d3e0 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40b2670 con 0x7f4ec4153660 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40b2950 con 0x7f4ec4153a30 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec415d3e0 0x7f4ec413a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec415d3e0 0x7f4ec413a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37410/0 (socket says 172.21.15.100:37410) 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 -- 172.21.15.100:0/673220323 learned_addr learned my addr 172.21.15.100:0/673220323 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4eca3b2700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4153660 0x7f4ec40b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ec9bb1700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ec4153a30 0x7f4ec415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 -- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ec4153a30 msgr2=0x7f4ec415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ec4153a30 0x7f4ec415f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 -- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4153660 msgr2=0x7f4ec40b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4153660 0x7f4ec40b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 -- 172.21.15.100:0/673220323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8009420 con 0x7f4ec415d3e0 2024-08-17T02:22:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4eca3b2700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4153660 0x7f4ec40b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecabb3700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec415d3e0 0x7f4ec413a9a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00ea50 tx=0x7f4ebc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ec27fc700 1 -- 172.21.15.100:0/673220323 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00c920 con 0x7f4ec415d3e0 2024-08-17T02:22:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ec27fc700 1 -- 172.21.15.100:0/673220323 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc00caa0 con 0x7f4ec415d3e0 2024-08-17T02:22:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec4157560 con 0x7f4ec415d3e0 2024-08-17T02:22:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ec27fc700 1 -- 172.21.15.100:0/673220323 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00ce60 con 0x7f4ec415d3e0 2024-08-17T02:22:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.716+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec40b2c30 con 0x7f4ec415d3e0 2024-08-17T02:22:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.720+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec40645c0 con 0x7f4ec415d3e0 2024-08-17T02:22:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.720+0000 7f4ec27fc700 1 -- 172.21.15.100:0/673220323 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4ebc013680 con 0x7f4ec415d3e0 2024-08-17T02:22:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.720+0000 7f4ec27fc700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4eb0060860 0x7f4eb0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.720+0000 7f4ec27fc700 1 -- 172.21.15.100:0/673220323 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ebc07cfd0 con 0x7f4ec415d3e0 2024-08-17T02:22:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.720+0000 7f4eca3b2700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4eb0060860 0x7f4eb0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.720+0000 7f4eca3b2700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4eb0060860 0x7f4eb0062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4005c50 tx=0x7f4eb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.728+0000 7f4ec27fc700 1 -- 172.21.15.100:0/673220323 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4ebc04d420 con 0x7f4ec415d3e0 2024-08-17T02:22:33.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ec40b3e90 con 0x7f4ec415d3e0 2024-08-17T02:22:33.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ec27fc700 1 -- 172.21.15.100:0/673220323 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ebc050a40 con 0x7f4ec415d3e0 2024-08-17T02:22:33.894 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:22:33.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4eb0060860 msgr2=0x7f4eb0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4eb0060860 0x7f4eb0062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4005c50 tx=0x7f4eb4009040 comp rx=0 tx=0).stop 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec415d3e0 msgr2=0x7f4ec413a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec415d3e0 0x7f4ec413a9a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00ea50 tx=0x7f4ebc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 shutdown_connections 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4eb0060860 0x7f4eb0062d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec415d3e0 0x7f4ec413a9a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ec4153a30 0x7f4ec415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 --2- 172.21.15.100:0/673220323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4153660 0x7f4ec40b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 >> 172.21.15.100:0/673220323 conn(0x7f4ec40caad0 msgr2=0x7f4ec4069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 shutdown_connections 2024-08-17T02:22:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:33.892+0000 7f4ecc616700 1 -- 172.21.15.100:0/673220323 wait complete. 2024-08-17T02:22:33.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:22:34.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.020+0000 7fddc886c700 1 Processor -- start 2024-08-17T02:22:34.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.020+0000 7fddc886c700 1 -- start start 2024-08-17T02:22:34.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fddc4067030 0x7fddc4067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fddc40cfb00 0x7fddc40cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40cf670 con 0x7fddc40cb5c0 2024-08-17T02:22:34.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc40cea60 con 0x7fddc4067030 2024-08-17T02:22:34.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc40ced40 con 0x7fddc40cfb00 2024-08-17T02:22:34.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc37fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fddc4067030 0x7fddc4067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc3fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fddc40cb5c0 0x7fddc4158cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58840/0 (socket says 172.21.15.100:58840) 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc37fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fddc4067030 0x7fddc4067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37418/0 (socket says 172.21.15.100:37418) 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc3fff700 1 -- 172.21.15.100:0/765168398 learned_addr learned my addr 172.21.15.100:0/765168398 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc2ffd700 1 --2- 172.21.15.100:0/765168398 >> v2:172.21.15.123:3300/0 conn(0x7fddc40cfb00 0x7fddc40cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc37fe700 1 -- 172.21.15.100:0/765168398 >> v2:172.21.15.123:3300/0 conn(0x7fddc40cfb00 msgr2=0x7fddc40cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc37fe700 1 --2- 172.21.15.100:0/765168398 >> v2:172.21.15.123:3300/0 conn(0x7fddc40cfb00 0x7fddc40cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc37fe700 1 -- 172.21.15.100:0/765168398 >> v1:172.21.15.100:6789/0 conn(0x7fddc40cb5c0 legacy=0x7fddc4158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc27fc700 1 -- 172.21.15.100:0/765168398 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918128712 0 0) 0x7fddc40cf670 con 0x7fddc40cb5c0 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc37fe700 1 -- 172.21.15.100:0/765168398 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40cf020 con 0x7fddc4067030 2024-08-17T02:22:34.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc37fe700 1 --2- 172.21.15.100:0/765168398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc4067400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fddc406fb70 tx=0x7fddb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79df76ee94b2337 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc27fc700 1 -- 172.21.15.100:0/765168398 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8011630 con 0x7fddc4067030 2024-08-17T02:22:34.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc27fc700 1 -- 172.21.15.100:0/765168398 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8011790 con 0x7fddc4067030 2024-08-17T02:22:34.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc27fc700 1 -- 172.21.15.100:0/765168398 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8011b30 con 0x7fddc4067030 2024-08-17T02:22:34.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 -- 172.21.15.100:0/765168398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 msgr2=0x7fddc4067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 --2- 172.21.15.100:0/765168398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc4067400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fddc406fb70 tx=0x7fddb80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 -- 172.21.15.100:0/765168398 shutdown_connections 2024-08-17T02:22:34.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 --2- 172.21.15.100:0/765168398 >> v2:172.21.15.123:3300/0 conn(0x7fddc40cfb00 0x7fddc40cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 --2- 172.21.15.100:0/765168398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc4067400 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 -- 172.21.15.100:0/765168398 >> 172.21.15.100:0/765168398 conn(0x7fddc40d0b50 msgr2=0x7fddc40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:34.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 -- 172.21.15.100:0/765168398 shutdown_connections 2024-08-17T02:22:34.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.024+0000 7fddc886c700 1 -- 172.21.15.100:0/765168398 wait complete. 2024-08-17T02:22:34.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 Processor -- start 2024-08-17T02:22:34.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 -- start start 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc416cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddc40cb5c0 0x7fddc40c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddc40cfb00 0x7fddc4169750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc415a820 con 0x7fddc4067030 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc415c7b0 con 0x7fddc40cfb00 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4159c10 con 0x7fddc40cb5c0 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc416cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc416cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37424/0 (socket says 172.21.15.100:37424) 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 -- 172.21.15.100:0/1224425578 learned_addr learned my addr 172.21.15.100:0/1224425578 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc3fff700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddc40cfb00 0x7fddc4169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc2ffd700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddc40cb5c0 0x7fddc40c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 -- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddc40cb5c0 msgr2=0x7fddc40c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddc40cb5c0 0x7fddc40c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 -- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddc40cfb00 msgr2=0x7fddc4169750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddc40cfb00 0x7fddc4169750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 -- 172.21.15.100:0/1224425578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8009420 con 0x7fddc4067030 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc37fe700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc416cf80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fddc4069520 tx=0x7fddb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddab7fe700 1 -- 172.21.15.100:0/1224425578 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8000d10 con 0x7fddc4067030 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddab7fe700 1 -- 172.21.15.100:0/1224425578 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8011b10 con 0x7fddc4067030 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc4159ef0 con 0x7fddc4067030 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddab7fe700 1 -- 172.21.15.100:0/1224425578 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8025640 con 0x7fddc4067030 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.028+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc415a4b0 con 0x7fddc4067030 2024-08-17T02:22:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.032+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc40638f0 con 0x7fddc4067030 2024-08-17T02:22:34.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.032+0000 7fddab7fe700 1 -- 172.21.15.100:0/1224425578 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fddb8025a30 con 0x7fddc4067030 2024-08-17T02:22:34.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.032+0000 7fddab7fe700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fddb0060860 0x7fddb0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.032+0000 7fddab7fe700 1 -- 172.21.15.100:0/1224425578 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fddb807e3f0 con 0x7fddc4067030 2024-08-17T02:22:34.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.032+0000 7fddc2ffd700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fddb0060860 0x7fddb0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.032+0000 7fddc2ffd700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fddb0060860 0x7fddb0062d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fddc415c200 tx=0x7fddb40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.040+0000 7fddab7fe700 1 -- 172.21.15.100:0/1224425578 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fddb804e840 con 0x7fddc4067030 2024-08-17T02:22:34.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fddc40b0d00 con 0x7fddc4067030 2024-08-17T02:22:34.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddab7fe700 1 -- 172.21.15.100:0/1224425578 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fddb8051e60 con 0x7fddc4067030 2024-08-17T02:22:34.210 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:22:34.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fddb0060860 msgr2=0x7fddb0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fddb0060860 0x7fddb0062d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fddc415c200 tx=0x7fddb40091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 msgr2=0x7fddc416cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc416cf80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fddc4069520 tx=0x7fddb8017000 comp rx=0 tx=0).stop 2024-08-17T02:22:34.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 shutdown_connections 2024-08-17T02:22:34.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fddb0060860 0x7fddb0062d10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddc40cfb00 0x7fddc4169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddc40cb5c0 0x7fddc40c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 --2- 172.21.15.100:0/1224425578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4067030 0x7fddc416cf80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 >> 172.21.15.100:0/1224425578 conn(0x7fddc40d0b50 msgr2=0x7fddc40b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:34.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 shutdown_connections 2024-08-17T02:22:34.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.208+0000 7fddc886c700 1 -- 172.21.15.100:0/1224425578 wait complete. 2024-08-17T02:22:34.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.328+0000 7f66f3fff700 1 Processor -- start 2024-08-17T02:22:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.328+0000 7f66f3fff700 1 -- start start 2024-08-17T02:22:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.328+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.328+0000 7f66f3fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f415c130 con 0x7f66f40cb000 2024-08-17T02:22:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.328+0000 7f66f3fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f415b520 con 0x7f66f413bed0 2024-08-17T02:22:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.328+0000 7f66f3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f415b340 con 0x7f66f415af20 2024-08-17T02:22:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f27fc700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f66f40cb000 0x7f66f415a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57846/0 (socket says 172.21.15.100:57846) 2024-08-17T02:22:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f27fc700 1 -- 172.21.15.100:0/3751218762 learned_addr learned my addr 172.21.15.100:0/3751218762 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f37fe700 1 --2- 172.21.15.100:0/3751218762 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f1ffb700 1 -- 172.21.15.100:0/3751218762 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258303378 0 0) 0x7f66f415c130 con 0x7f66f40cb000 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f1ffb700 1 -- 172.21.15.100:0/3751218762 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66dc003570 con 0x7f66f40cb000 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f1ffb700 1 -- 172.21.15.100:0/3751218762 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570007097 0 0) 0x7f66f415b520 con 0x7f66f413bed0 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f1ffb700 1 -- 172.21.15.100:0/3751218762 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f415c130 con 0x7f66f413bed0 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f37fe700 1 -- 172.21.15.100:0/3751218762 >> v1:172.21.15.123:6789/0 conn(0x7f66f413bed0 legacy=0x7f66f40ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f37fe700 1 -- 172.21.15.100:0/3751218762 >> v1:172.21.15.100:6789/0 conn(0x7f66f40cb000 legacy=0x7f66f415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f1ffb700 1 -- 172.21.15.100:0/3751218762 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067843839 0 0) 0x7f66dc003570 con 0x7f66f40cb000 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f37fe700 1 -- 172.21.15.100:0/3751218762 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f415b800 con 0x7f66f415af20 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f37fe700 1 --2- 172.21.15.100:0/3751218762 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f415c5c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009e90 tx=0x7f66ec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d054fed936fe1be9 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f1ffb700 1 -- 172.21.15.100:0/3751218762 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00e070 con 0x7f66f415af20 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f1ffb700 1 -- 172.21.15.100:0/3751218762 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec00bb40 con 0x7f66f415af20 2024-08-17T02:22:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f1ffb700 1 -- 172.21.15.100:0/3751218762 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00bec0 con 0x7f66f415af20 2024-08-17T02:22:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f3fff700 1 -- 172.21.15.100:0/3751218762 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 msgr2=0x7f66f415c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f3fff700 1 --2- 172.21.15.100:0/3751218762 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f415c5c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009e90 tx=0x7f66ec009780 comp rx=0 tx=0).stop 2024-08-17T02:22:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f3fff700 1 -- 172.21.15.100:0/3751218762 shutdown_connections 2024-08-17T02:22:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f3fff700 1 --2- 172.21.15.100:0/3751218762 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f415c5c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f3fff700 1 -- 172.21.15.100:0/3751218762 >> 172.21.15.100:0/3751218762 conn(0x7f66f406e100 msgr2=0x7f66f40b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f3fff700 1 -- 172.21.15.100:0/3751218762 shutdown_connections 2024-08-17T02:22:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.332+0000 7f66f3fff700 1 -- 172.21.15.100:0/3751218762 wait complete. 2024-08-17T02:22:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f3fff700 1 Processor -- start 2024-08-17T02:22:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f3fff700 1 -- start start 2024-08-17T02:22:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f40cb000 0x7f66f414ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f413bed0 0x7f66f412e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f414e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f3fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f414ef20 con 0x7f66f40cb000 2024-08-17T02:22:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f414f200 con 0x7f66f413bed0 2024-08-17T02:22:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f414f4e0 con 0x7f66f415af20 2024-08-17T02:22:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f40cb000 0x7f66f414ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f414e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f40cb000 0x7f66f414ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49980/0 (socket says 172.21.15.100:49980) 2024-08-17T02:22:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f413bed0 0x7f66f412e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 -- 172.21.15.100:0/515870867 learned_addr learned my addr 172.21.15.100:0/515870867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 -- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 msgr2=0x7f66f414e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f414e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 -- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f413bed0 msgr2=0x7f66f412e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f413bed0 0x7f66f412e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 -- 172.21.15.100:0/515870867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec009a20 con 0x7f66f40cb000 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f27fc700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f413bed0 0x7f66f412e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.336+0000 7f66f2ffd700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f40cb000 0x7f66f414ffc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c530 tx=0x7f66e400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66daffd700 1 -- 172.21.15.100:0/515870867 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011590 con 0x7f66f40cb000 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66daffd700 1 -- 172.21.15.100:0/515870867 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4011710 con 0x7f66f40cb000 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f414f7c0 con 0x7f66f40cb000 2024-08-17T02:22:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66daffd700 1 -- 172.21.15.100:0/515870867 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011ad0 con 0x7f66f40cb000 2024-08-17T02:22:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f412eaf0 con 0x7f66f40cb000 2024-08-17T02:22:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66daffd700 1 -- 172.21.15.100:0/515870867 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f66e401b3b0 con 0x7f66f40cb000 2024-08-17T02:22:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f414fcc0 con 0x7f66f40cb000 2024-08-17T02:22:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66daffd700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66e0060860 0x7f66e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66daffd700 1 -- 172.21.15.100:0/515870867 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f66e407bfb0 con 0x7f66f40cb000 2024-08-17T02:22:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.340+0000 7f66f27fc700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66e0060860 0x7f66e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.344+0000 7f66f27fc700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66e0060860 0x7f66e0062d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f66f4149880 tx=0x7f66e8007d10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.352+0000 7f66daffd700 1 -- 172.21.15.100:0/515870867 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f66e404c400 con 0x7f66f40cb000 2024-08-17T02:22:34.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.520+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66f40638f0 con 0x7f66f40cb000 2024-08-17T02:22:34.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.520+0000 7f66daffd700 1 -- 172.21.15.100:0/515870867 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f66e4017020 con 0x7f66f40cb000 2024-08-17T02:22:34.524 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:34.524 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:34.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66e0060860 msgr2=0x7f66e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66e0060860 0x7f66e0062d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f66f4149880 tx=0x7f66e8007d10 comp rx=0 tx=0).stop 2024-08-17T02:22:34.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f40cb000 msgr2=0x7f66f414ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f40cb000 0x7f66f414ffc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c530 tx=0x7f66e400c840 comp rx=0 tx=0).stop 2024-08-17T02:22:34.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 shutdown_connections 2024-08-17T02:22:34.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66e0060860 0x7f66e0062d10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f415af20 0x7f66f414e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f413bed0 0x7f66f412e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 --2- 172.21.15.100:0/515870867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f40cb000 0x7f66f414ffc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 >> 172.21.15.100:0/515870867 conn(0x7f66f406e100 msgr2=0x7f66f4156910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:34.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 shutdown_connections 2024-08-17T02:22:34.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.524+0000 7f66f3fff700 1 -- 172.21.15.100:0/515870867 wait complete. 2024-08-17T02:22:34.539 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:02.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:03.458625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:22:34.539 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.648+0000 7f6e032aa700 1 Processor -- start 2024-08-17T02:22:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.648+0000 7f6e032aa700 1 -- start start 2024-08-17T02:22:34.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.648+0000 7f6e032aa700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6dfc067510 0x7f6dfc0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.648+0000 7f6e032aa700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0ce210 con 0x7f6dfc159480 2024-08-17T02:22:34.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.648+0000 7f6e032aa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0cb440 con 0x7f6dfc1590b0 2024-08-17T02:22:34.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.648+0000 7f6e032aa700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc0cba00 con 0x7f6dfc067510 2024-08-17T02:22:34.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e02aa9700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6dfc159480 0x7f6dfc0cebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57858/0 (socket says 172.21.15.100:57858) 2024-08-17T02:22:34.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e02aa9700 1 -- 172.21.15.100:0/4212107484 learned_addr learned my addr 172.21.15.100:0/4212107484 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:34.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e01aa7700 1 --2- 172.21.15.100:0/4212107484 >> v2:172.21.15.123:3300/0 conn(0x7f6dfc067510 0x7f6dfc0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e012a6700 1 -- 172.21.15.100:0/4212107484 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275172860 0 0) 0x7f6dfc0ce210 con 0x7f6dfc159480 2024-08-17T02:22:34.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e012a6700 1 -- 172.21.15.100:0/4212107484 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de8003570 con 0x7f6dfc159480 2024-08-17T02:22:34.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e012a6700 1 -- 172.21.15.100:0/4212107484 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212424742 0 0) 0x7f6de8003570 con 0x7f6dfc159480 2024-08-17T02:22:34.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e01aa7700 1 -- 172.21.15.100:0/4212107484 >> v1:172.21.15.123:6789/0 conn(0x7f6dfc1590b0 legacy=0x7f6dfc066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e01aa7700 1 -- 172.21.15.100:0/4212107484 >> v1:172.21.15.100:6789/0 conn(0x7f6dfc159480 legacy=0x7f6dfc0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e01aa7700 1 -- 172.21.15.100:0/4212107484 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0cb720 con 0x7f6dfc067510 2024-08-17T02:22:34.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e01aa7700 1 --2- 172.21.15.100:0/4212107484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 0x7f6dfc0ce6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6df80098e0 tx=0x7f6df80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ecf6a2e4b4b826 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e012a6700 1 -- 172.21.15.100:0/4212107484 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df40017f0 con 0x7f6dfc067510 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e012a6700 1 -- 172.21.15.100:0/4212107484 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df8011420 con 0x7f6dfc067510 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e012a6700 1 -- 172.21.15.100:0/4212107484 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80117c0 con 0x7f6dfc067510 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e032aa700 1 -- 172.21.15.100:0/4212107484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 msgr2=0x7f6dfc0ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e032aa700 1 --2- 172.21.15.100:0/4212107484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 0x7f6dfc0ce6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6df80098e0 tx=0x7f6df80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e032aa700 1 -- 172.21.15.100:0/4212107484 shutdown_connections 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e032aa700 1 --2- 172.21.15.100:0/4212107484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 0x7f6dfc0ce6a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e032aa700 1 -- 172.21.15.100:0/4212107484 >> 172.21.15.100:0/4212107484 conn(0x7f6dfc0d0b90 msgr2=0x7f6dfc0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e032aa700 1 -- 172.21.15.100:0/4212107484 shutdown_connections 2024-08-17T02:22:34.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.652+0000 7f6e032aa700 1 -- 172.21.15.100:0/4212107484 wait complete. 2024-08-17T02:22:34.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e032aa700 1 Processor -- start 2024-08-17T02:22:34.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e032aa700 1 -- start start 2024-08-17T02:22:34.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e032aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 0x7f6dfc16d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e032aa700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dfc1590b0 0x7f6dfc151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e032aa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc159480 0x7f6dfc15aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e032aa700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc1549f0 con 0x7f6dfc1590b0 2024-08-17T02:22:34.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e032aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc16db80 con 0x7f6dfc067510 2024-08-17T02:22:34.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e032aa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc151a90 con 0x7f6dfc159480 2024-08-17T02:22:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dfc1590b0 0x7f6dfc151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dfc1590b0 0x7f6dfc151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49986/0 (socket says 172.21.15.100:49986) 2024-08-17T02:22:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 -- 172.21.15.100:0/2697824054 learned_addr learned my addr 172.21.15.100:0/2697824054 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e022a8700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 0x7f6dfc16d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e02aa9700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc159480 0x7f6dfc15aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 -- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc159480 msgr2=0x7f6dfc15aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc159480 0x7f6dfc15aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 -- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 msgr2=0x7f6dfc16d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 0x7f6dfc16d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 -- 172.21.15.100:0/2697824054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8009420 con 0x7f6dfc1590b0 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e022a8700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 0x7f6dfc16d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e02aa9700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc159480 0x7f6dfc15aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.656+0000 7f6e01aa7700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dfc1590b0 0x7f6dfc151550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc069480 tx=0x7f6df8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6df27fc700 1 -- 172.21.15.100:0/2697824054 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df801cc90 con 0x7f6dfc1590b0 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6df27fc700 1 -- 172.21.15.100:0/2697824054 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df801cdf0 con 0x7f6dfc1590b0 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc0c86c0 con 0x7f6dfc1590b0 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6df27fc700 1 -- 172.21.15.100:0/2697824054 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df801a6a0 con 0x7f6dfc1590b0 2024-08-17T02:22:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc0c7ab0 con 0x7f6dfc1590b0 2024-08-17T02:22:34.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6df27fc700 1 -- 172.21.15.100:0/2697824054 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6df801aa50 con 0x7f6dfc1590b0 2024-08-17T02:22:34.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc064330 con 0x7f6dfc1590b0 2024-08-17T02:22:34.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6df27fc700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6de0060860 0x7f6de0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6df27fc700 1 -- 172.21.15.100:0/2697824054 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6df807e560 con 0x7f6dfc1590b0 2024-08-17T02:22:34.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.660+0000 7f6e022a8700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6de0060860 0x7f6de0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.664+0000 7f6e022a8700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6de0060860 0x7f6de0062d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6dec007fd0 tx=0x7f6dec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.668+0000 7f6df27fc700 1 -- 172.21.15.100:0/2697824054 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6df804e9b0 con 0x7f6dfc1590b0 2024-08-17T02:22:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.828+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6dfc070cf0 con 0x7f6dfc1590b0 2024-08-17T02:22:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6df27fc700 1 -- 172.21.15.100:0/2697824054 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6df8051fd0 con 0x7f6dfc1590b0 2024-08-17T02:22:34.833 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:34.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6de0060860 msgr2=0x7f6de0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6de0060860 0x7f6de0062d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6dec007fd0 tx=0x7f6dec009040 comp rx=0 tx=0).stop 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dfc1590b0 msgr2=0x7f6dfc151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dfc1590b0 0x7f6dfc151550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc069480 tx=0x7f6df8016000 comp rx=0 tx=0).stop 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 shutdown_connections 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6de0060860 0x7f6de0062d10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc159480 0x7f6dfc15aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dfc1590b0 0x7f6dfc151550 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 --2- 172.21.15.100:0/2697824054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc067510 0x7f6dfc16d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 >> 172.21.15.100:0/2697824054 conn(0x7f6dfc0d0b90 msgr2=0x7f6dfc15ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 shutdown_connections 2024-08-17T02:22:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.832+0000 7f6e032aa700 1 -- 172.21.15.100:0/2697824054 wait complete. 2024-08-17T02:22:34.847 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:22:34.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 Processor -- start 2024-08-17T02:22:34.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- start start 2024-08-17T02:22:34.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9338146f40 0x7f933806e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f933815dcc0 0x7f933806ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338147310 0x7f93380a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93380b0010 con 0x7f933815dcc0 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93380af440 con 0x7f9338146f40 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93380af6e0 con 0x7f9338147310 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f933815dcc0 0x7f933806ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f933815dcc0 0x7f933806ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49992/0 (socket says 172.21.15.100:49992) 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 -- 172.21.15.100:0/1652838103 learned_addr learned my addr 172.21.15.100:0/1652838103 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9337fff700 1 --2- 172.21.15.100:0/1652838103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338147310 0x7f93380a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f93377fe700 1 --2- 172.21.15.100:0/1652838103 >> v2:172.21.15.123:3300/0 conn(0x7f9338146f40 0x7f933806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 -- 172.21.15.100:0/1652838103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338147310 msgr2=0x7f93380a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 --2- 172.21.15.100:0/1652838103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338147310 0x7f93380a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 -- 172.21.15.100:0/1652838103 >> v2:172.21.15.123:3300/0 conn(0x7f9338146f40 msgr2=0x7f933806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 --2- 172.21.15.100:0/1652838103 >> v2:172.21.15.123:3300/0 conn(0x7f9338146f40 0x7f933806e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 -- 172.21.15.100:0/1652838103 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380af9c0 con 0x7f933815dcc0 2024-08-17T02:22:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f93377fe700 1 --2- 172.21.15.100:0/1652838103 >> v2:172.21.15.123:3300/0 conn(0x7f9338146f40 0x7f933806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f9336ffd700 1 --2- 172.21.15.100:0/1652838103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806ea80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f93380657b0 tx=0x7f9330009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bfd35fdca0b6ee server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f93367fc700 1 -- 172.21.15.100:0/1652838103 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933000e070 con 0x7f933815dcc0 2024-08-17T02:22:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f93367fc700 1 -- 172.21.15.100:0/1652838103 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933000ba20 con 0x7f933815dcc0 2024-08-17T02:22:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f93367fc700 1 -- 172.21.15.100:0/1652838103 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300092e0 con 0x7f933815dcc0 2024-08-17T02:22:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- 172.21.15.100:0/1652838103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 msgr2=0x7f933806ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 --2- 172.21.15.100:0/1652838103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806ea80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f93380657b0 tx=0x7f9330009840 comp rx=0 tx=0).stop 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- 172.21.15.100:0/1652838103 shutdown_connections 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 --2- 172.21.15.100:0/1652838103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338147310 0x7f93380a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 --2- 172.21.15.100:0/1652838103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806ea80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 --2- 172.21.15.100:0/1652838103 >> v2:172.21.15.123:3300/0 conn(0x7f9338146f40 0x7f933806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- 172.21.15.100:0/1652838103 >> 172.21.15.100:0/1652838103 conn(0x7f9338066e00 msgr2=0x7f9338067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- 172.21.15.100:0/1652838103 shutdown_connections 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.952+0000 7f933e1c2700 1 -- 172.21.15.100:0/1652838103 wait complete. 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f933e1c2700 1 Processor -- start 2024-08-17T02:22:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f933e1c2700 1 -- start start 2024-08-17T02:22:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f933e1c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338146f40 0x7f93381425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f933e1c2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338147310 0x7f93380c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f933e1c2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f933e1c2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338096d60 con 0x7f933815dcc0 2024-08-17T02:22:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f933e1c2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93380c8e40 con 0x7f9338147310 2024-08-17T02:22:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f933e1c2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93380c9120 con 0x7f9338146f40 2024-08-17T02:22:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50000/0 (socket says 172.21.15.100:50000) 2024-08-17T02:22:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 -- 172.21.15.100:0/3908228221 learned_addr learned my addr 172.21.15.100:0/3908228221 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9336ffd700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338147310 0x7f93380c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f93377fe700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338146f40 0x7f93381425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 -- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338146f40 msgr2=0x7f93381425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338146f40 0x7f93381425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 -- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338147310 msgr2=0x7f93380c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338147310 0x7f93380c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 -- 172.21.15.100:0/3908228221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330009ae0 con 0x7f933815dcc0 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f9337fff700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806a010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f932800ea50 tx=0x7f93280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.956+0000 7f931f7fe700 1 -- 172.21.15.100:0/3908228221 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932800ee90 con 0x7f933815dcc0 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f931f7fe700 1 -- 172.21.15.100:0/3908228221 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932800c850 con 0x7f933815dcc0 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93380c9400 con 0x7f933815dcc0 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f931f7fe700 1 -- 172.21.15.100:0/3908228221 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932800cc10 con 0x7f933815dcc0 2024-08-17T02:22:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9338142b20 con 0x7f933815dcc0 2024-08-17T02:22:34.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9338096ed0 con 0x7f933815dcc0 2024-08-17T02:22:34.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f931f7fe700 1 -- 172.21.15.100:0/3908228221 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9328013470 con 0x7f933815dcc0 2024-08-17T02:22:34.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f931f7fe700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9324060890 0x7f9324062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:34.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f931f7fe700 1 -- 172.21.15.100:0/3908228221 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f932807cd30 con 0x7f933815dcc0 2024-08-17T02:22:34.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f93377fe700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9324060890 0x7f9324062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.960+0000 7f93377fe700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9324060890 0x7f9324062d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f932c006f00 tx=0x7f932c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:34.968+0000 7f931f7fe700 1 -- 172.21.15.100:0/3908228221 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f932804d180 con 0x7f933815dcc0 2024-08-17T02:22:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.128+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93380645c0 con 0x7f933815dcc0 2024-08-17T02:22:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.128+0000 7f931f7fe700 1 -- 172.21.15.100:0/3908228221 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93280507a0 con 0x7f933815dcc0 2024-08-17T02:22:35.132 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:22:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9324060890 msgr2=0x7f9324062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9324060890 0x7f9324062d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f932c006f00 tx=0x7f932c008040 comp rx=0 tx=0).stop 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 msgr2=0x7f933806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806a010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f932800ea50 tx=0x7f93280091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 shutdown_connections 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9324060890 0x7f9324062d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933815dcc0 0x7f933806a010 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338147310 0x7f93380c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 --2- 172.21.15.100:0/3908228221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9338146f40 0x7f93381425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 >> 172.21.15.100:0/3908228221 conn(0x7f9338066e00 msgr2=0x7f93380a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 shutdown_connections 2024-08-17T02:22:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.132+0000 7f933e1c2700 1 -- 172.21.15.100:0/3908228221 wait complete. 2024-08-17T02:22:35.145 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:22:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.248+0000 7fbca01f8700 1 Processor -- start 2024-08-17T02:22:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.248+0000 7fbca01f8700 1 -- start start 2024-08-17T02:22:35.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.248+0000 7fbca01f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc981596d0 0x7fbc98066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.248+0000 7fbca01f8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc9815dbc0 0x7fbc98067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.248+0000 7fbca01f8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc980cba90 con 0x7fbc98159300 2024-08-17T02:22:35.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.248+0000 7fbca01f8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc980b2b60 con 0x7fbc9815dbc0 2024-08-17T02:22:35.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.248+0000 7fbca01f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980b2940 con 0x7fbc981596d0 2024-08-17T02:22:35.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9e795700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc9815dbc0 0x7fbc98067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9d793700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc981596d0 0x7fbc98066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9e795700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc9815dbc0 0x7fbc98067340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:41430/0 (socket says 172.21.15.100:41430) 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9df94700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbc98159300 0x7fbc980b2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:58220/0 (socket says 172.21.15.100:58220) 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9d793700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc981596d0 0x7fbc98066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:48970/0 (socket says 172.21.15.100:48970) 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9e795700 1 -- 172.21.15.100:0/2749814520 learned_addr learned my addr 172.21.15.100:0/2749814520 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9e795700 1 -- 172.21.15.100:0/2749814520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc981596d0 msgr2=0x7fbc98066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9e795700 1 --2- 172.21.15.100:0/2749814520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc981596d0 0x7fbc98066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9e795700 1 -- 172.21.15.100:0/2749814520 >> v1:172.21.15.123:6789/0 conn(0x7fbc98159300 legacy=0x7fbc980b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9e795700 1 -- 172.21.15.100:0/2749814520 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc980cb160 con 0x7fbc9815dbc0 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9d793700 1 --2- 172.21.15.100:0/2749814520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc981596d0 0x7fbc98066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:35.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9e795700 1 --2- 172.21.15.100:0/2749814520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815dbc0 0x7fbc98067340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009f20 tx=0x7fbc90009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fd419069c1fee74 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9cf92700 1 -- 172.21.15.100:0/2749814520 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000e070 con 0x7fbc9815dbc0 2024-08-17T02:22:35.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9cf92700 1 -- 172.21.15.100:0/2749814520 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9000b8d0 con 0x7fbc9815dbc0 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbc9cf92700 1 -- 172.21.15.100:0/2749814520 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900092e0 con 0x7fbc9815dbc0 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbca01f8700 1 -- 172.21.15.100:0/2749814520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815dbc0 msgr2=0x7fbc98067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2749814520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815dbc0 0x7fbc98067340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009f20 tx=0x7fbc90009840 comp rx=0 tx=0).stop 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbca01f8700 1 -- 172.21.15.100:0/2749814520 shutdown_connections 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2749814520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815dbc0 0x7fbc98067340 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2749814520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc981596d0 0x7fbc98066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbca01f8700 1 -- 172.21.15.100:0/2749814520 >> 172.21.15.100:0/2749814520 conn(0x7fbc980682c0 msgr2=0x7fbc980686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbca01f8700 1 -- 172.21.15.100:0/2749814520 shutdown_connections 2024-08-17T02:22:35.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.252+0000 7fbca01f8700 1 -- 172.21.15.100:0/2749814520 wait complete. 2024-08-17T02:22:35.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 Processor -- start 2024-08-17T02:22:35.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 -- start start 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc98159300 0x7fbc980c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc981596d0 0x7fbc980c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9815dbc0 0x7fbc980b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc981699c0 con 0x7fbc9815dbc0 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980c6540 con 0x7fbc981596d0 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980c46a0 con 0x7fbc98159300 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9815dbc0 0x7fbc980b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9d793700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc981596d0 0x7fbc980c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9815dbc0 0x7fbc980b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50008/0 (socket says 172.21.15.100:50008) 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 -- 172.21.15.100:0/2874963817 learned_addr learned my addr 172.21.15.100:0/2874963817 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9df94700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc98159300 0x7fbc980c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 -- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc98159300 msgr2=0x7fbc980c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc98159300 0x7fbc980c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 -- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc981596d0 msgr2=0x7fbc980c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc981596d0 0x7fbc980c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 -- 172.21.15.100:0/2874963817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc94009420 con 0x7fbc9815dbc0 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc9e795700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9815dbc0 0x7fbc980b3000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cde20 tx=0x7fbc90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc8dffb700 1 -- 172.21.15.100:0/2874963817 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90012790 con 0x7fbc9815dbc0 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc8dffb700 1 -- 172.21.15.100:0/2874963817 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc900128f0 con 0x7fbc9815dbc0 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc90009ae0 con 0x7fbc9815dbc0 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbc8dffb700 1 -- 172.21.15.100:0/2874963817 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90012c90 con 0x7fbc9815dbc0 2024-08-17T02:22:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.256+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc980c4c60 con 0x7fbc9815dbc0 2024-08-17T02:22:35.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.260+0000 7fbc8dffb700 1 -- 172.21.15.100:0/2874963817 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbc9001d630 con 0x7fbc9815dbc0 2024-08-17T02:22:35.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.260+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc98169b70 con 0x7fbc9815dbc0 2024-08-17T02:22:35.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.260+0000 7fbc8dffb700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbc7c060860 0x7fbc7c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.260+0000 7fbc8dffb700 1 -- 172.21.15.100:0/2874963817 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc9007e000 con 0x7fbc9815dbc0 2024-08-17T02:22:35.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.260+0000 7fbc9df94700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbc7c060860 0x7fbc7c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.260+0000 7fbc9df94700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbc7c060860 0x7fbc7c062d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc880010e0 tx=0x7fbc88007e70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.268+0000 7fbc8dffb700 1 -- 172.21.15.100:0/2874963817 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbc98169b70 con 0x7fbc9815dbc0 2024-08-17T02:22:35.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbc980645c0 con 0x7fbc9815dbc0 2024-08-17T02:22:35.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbc8dffb700 1 -- 172.21.15.100:0/2874963817 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbc9004e450 con 0x7fbc9815dbc0 2024-08-17T02:22:35.437 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:22:35.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbc7c060860 msgr2=0x7fbc7c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbc7c060860 0x7fbc7c062d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc880010e0 tx=0x7fbc88007e70 comp rx=0 tx=0).stop 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9815dbc0 msgr2=0x7fbc980b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9815dbc0 0x7fbc980b3000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cde20 tx=0x7fbc90017000 comp rx=0 tx=0).stop 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 shutdown_connections 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbc7c060860 0x7fbc7c062d10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9815dbc0 0x7fbc980b3000 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc981596d0 0x7fbc980c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 --2- 172.21.15.100:0/2874963817 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc98159300 0x7fbc980c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 >> 172.21.15.100:0/2874963817 conn(0x7fbc980682c0 msgr2=0x7fbc980b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 shutdown_connections 2024-08-17T02:22:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.436+0000 7fbca01f8700 1 -- 172.21.15.100:0/2874963817 wait complete. 2024-08-17T02:22:35.452 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0ac32700 1 Processor -- start 2024-08-17T02:22:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0ac32700 1 -- start start 2024-08-17T02:22:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0ac32700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda040b14a0 0x7fda040cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0ac32700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda040cbaa0 con 0x7fda040cb390 2024-08-17T02:22:35.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0ac32700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda04157630 con 0x7fda04154900 2024-08-17T02:22:35.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0ac32700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda04068790 con 0x7fda040b14a0 2024-08-17T02:22:35.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fda040cb390 0x7fda04160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57864/0 (socket says 172.21.15.100:57864) 2024-08-17T02:22:35.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 -- 172.21.15.100:0/882588653 learned_addr learned my addr 172.21.15.100:0/882588653 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:35.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0942f700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda040b14a0 0x7fda040cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261064631 0 0) 0x7fda040cbaa0 con 0x7fda040cb390 2024-08-17T02:22:35.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ec003570 con 0x7fda040cb390 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2915198013 0 0) 0x7fd9ec003570 con 0x7fda040cb390 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda040b14a0 msgr2=0x7fda040cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda040b14a0 0x7fda040cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 >> v1:172.21.15.123:6789/0 conn(0x7fda04154900 legacy=0x7fda040b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04157910 con 0x7fda040cb390 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7fd9fc001470 con 0x7fda040cb390 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 >> v1:172.21.15.100:6789/0 conn(0x7fda040cb390 legacy=0x7fda04160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda040b14a0 0x7fd9ec003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec004010 0x7fd9ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9ec004d50 0x7fd9ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04068790 con 0x7fd9ec004d50 2024-08-17T02:22:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec00bbf0 con 0x7fd9ec004010 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec00bdb0 con 0x7fda040b14a0 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9ec004d50 0x7fd9ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda09c30700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda040b14a0 0x7fd9ec003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0942f700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec004010 0x7fd9ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 -- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda040b14a0 msgr2=0x7fd9ec003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda040b14a0 0x7fd9ec003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 -- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec004010 msgr2=0x7fd9ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec004010 0x7fd9ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 -- 172.21.15.100:0/882588653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec003570 con 0x7fd9ec004d50 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.556+0000 7fda0a431700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9ec004d50 0x7fd9ec0051c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c390 tx=0x7fd9fc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b89a63325d82c5a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00cc60 con 0x7fd9ec004d50 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc00cdc0 con 0x7fd9ec004d50 2024-08-17T02:22:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda08c2e700 1 -- 172.21.15.100:0/882588653 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc005b70 con 0x7fd9ec004d50 2024-08-17T02:22:35.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 -- 172.21.15.100:0/882588653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9ec004d50 msgr2=0x7fd9ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 --2- 172.21.15.100:0/882588653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9ec004d50 0x7fd9ec0051c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c390 tx=0x7fd9fc001e50 comp rx=0 tx=0).stop 2024-08-17T02:22:35.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0a431700 1 -- 172.21.15.100:0/882588653 reap_dead start 2024-08-17T02:22:35.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 -- 172.21.15.100:0/882588653 shutdown_connections 2024-08-17T02:22:35.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 -- 172.21.15.100:0/882588653 >> 172.21.15.100:0/882588653 conn(0x7fda040678d0 msgr2=0x7fda04067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:35.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 -- 172.21.15.100:0/882588653 shutdown_connections 2024-08-17T02:22:35.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 -- 172.21.15.100:0/882588653 wait complete. 2024-08-17T02:22:35.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 Processor -- start 2024-08-17T02:22:35.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 -- start start 2024-08-17T02:22:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.560+0000 7fda0ac32700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda0414d710 0x7fda04159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0ac32700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda0414dae0 0x7fda04167620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0ac32700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0415b130 0x7fda0415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0ac32700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04167b60 con 0x7fda0415b130 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0ac32700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0406cfb0 con 0x7fda0414dae0 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0ac32700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0406d810 con 0x7fda0414d710 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0415b130 0x7fda0415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0415b130 0x7fda0415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50016/0 (socket says 172.21.15.100:50016) 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 -- 172.21.15.100:0/3731722649 learned_addr learned my addr 172.21.15.100:0/3731722649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0942f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda0414dae0 0x7fda04167620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda09c30700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda0414d710 0x7fda04159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 -- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda0414d710 msgr2=0x7fda04159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda0414d710 0x7fda04159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 -- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda0414dae0 msgr2=0x7fda04167620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda0414dae0 0x7fda04167620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 -- 172.21.15.100:0/3731722649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0415b940 con 0x7fda0415b130 2024-08-17T02:22:35.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0a431700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0415b130 0x7fda0415bd10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0057d0 tx=0x7fd9fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fd9f9ffb700 1 -- 172.21.15.100:0/3731722649 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc022070 con 0x7fda0415b130 2024-08-17T02:22:35.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fd9f9ffb700 1 -- 172.21.15.100:0/3731722649 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc000c30 con 0x7fda0415b130 2024-08-17T02:22:35.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0406d1c0 con 0x7fda0415b130 2024-08-17T02:22:35.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fd9f9ffb700 1 -- 172.21.15.100:0/3731722649 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00cc60 con 0x7fda0415b130 2024-08-17T02:22:35.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0415b5a0 con 0x7fda0415b130 2024-08-17T02:22:35.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0406d9c0 con 0x7fda0415b130 2024-08-17T02:22:35.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fd9f9ffb700 1 -- 172.21.15.100:0/3731722649 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd9fc007570 con 0x7fda0415b130 2024-08-17T02:22:35.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fd9f9ffb700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd9f0060860 0x7fd9f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.564+0000 7fd9f9ffb700 1 -- 172.21.15.100:0/3731722649 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd9fc07da00 con 0x7fda0415b130 2024-08-17T02:22:35.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.568+0000 7fda09c30700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd9f0060860 0x7fd9f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.568+0000 7fda09c30700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd9f0060860 0x7fd9f0062d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000c00 tx=0x7fd9f4007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.576+0000 7fd9f9ffb700 1 -- 172.21.15.100:0/3731722649 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fda0406d9c0 con 0x7fda0415b130 2024-08-17T02:22:35.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.732+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda0406d980 con 0x7fda0415b130 2024-08-17T02:22:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.732+0000 7fd9f9ffb700 1 -- 172.21.15.100:0/3731722649 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fd9fc017020 con 0x7fda0415b130 2024-08-17T02:22:35.735 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:35.735 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:35.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd9f0060860 msgr2=0x7fd9f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd9f0060860 0x7fd9f0062d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000c00 tx=0x7fd9f4007d30 comp rx=0 tx=0).stop 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0415b130 msgr2=0x7fda0415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0415b130 0x7fda0415bd10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0057d0 tx=0x7fd9fc015000 comp rx=0 tx=0).stop 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 shutdown_connections 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd9f0060860 0x7fd9f0062d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0415b130 0x7fda0415bd10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda0414dae0 0x7fda04167620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 --2- 172.21.15.100:0/3731722649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda0414d710 0x7fda04159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 >> 172.21.15.100:0/3731722649 conn(0x7fda040678d0 msgr2=0x7fda04063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 shutdown_connections 2024-08-17T02:22:35.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.736+0000 7fda0ac32700 1 -- 172.21.15.100:0/3731722649 wait complete. 2024-08-17T02:22:35.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:02.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:03.458625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:22:35.750 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.860+0000 7fb677fff700 1 Processor -- start 2024-08-17T02:22:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.860+0000 7fb677fff700 1 -- start start 2024-08-17T02:22:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb678140910 0x7fb6780c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6780cbab0 0x7fb6780cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6780cc450 con 0x7fb6780c2c80 2024-08-17T02:22:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6780c3e30 con 0x7fb6780cbab0 2024-08-17T02:22:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6780c4110 con 0x7fb678140910 2024-08-17T02:22:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb6767fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6780cbab0 0x7fb6780cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb6767fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6780cbab0 0x7fb6780cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50026/0 (socket says 172.21.15.100:50026) 2024-08-17T02:22:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb6767fc700 1 -- 172.21.15.100:0/3151468225 learned_addr learned my addr 172.21.15.100:0/3151468225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb676ffd700 1 --2- 172.21.15.100:0/3151468225 >> v2:172.21.15.123:3300/0 conn(0x7fb678140910 0x7fb6780c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb6767fc700 1 -- 172.21.15.100:0/3151468225 >> v2:172.21.15.123:3300/0 conn(0x7fb678140910 msgr2=0x7fb6780c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb6767fc700 1 --2- 172.21.15.100:0/3151468225 >> v2:172.21.15.123:3300/0 conn(0x7fb678140910 0x7fb6780c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb6767fc700 1 -- 172.21.15.100:0/3151468225 >> v1:172.21.15.100:6789/0 conn(0x7fb6780c2c80 legacy=0x7fb6780c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb675ffb700 1 -- 172.21.15.100:0/3151468225 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752185037 0 0) 0x7fb670001490 con 0x7fb6780c2c80 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb6767fc700 1 -- 172.21.15.100:0/3151468225 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780c43f0 con 0x7fb6780cbab0 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb6767fc700 1 --2- 172.21.15.100:0/3151468225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780cbf10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb67806e260 tx=0x7fb668009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60538c6699044b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb675ffb700 1 -- 172.21.15.100:0/3151468225 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800b860 con 0x7fb6780cbab0 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb675ffb700 1 -- 172.21.15.100:0/3151468225 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66800b9c0 con 0x7fb6780cbab0 2024-08-17T02:22:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb675ffb700 1 -- 172.21.15.100:0/3151468225 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800bd60 con 0x7fb6780cbab0 2024-08-17T02:22:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 -- 172.21.15.100:0/3151468225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 msgr2=0x7fb6780cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 --2- 172.21.15.100:0/3151468225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780cbf10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb67806e260 tx=0x7fb668009250 comp rx=0 tx=0).stop 2024-08-17T02:22:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 -- 172.21.15.100:0/3151468225 shutdown_connections 2024-08-17T02:22:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 --2- 172.21.15.100:0/3151468225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780cbf10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 --2- 172.21.15.100:0/3151468225 >> v2:172.21.15.123:3300/0 conn(0x7fb678140910 0x7fb6780c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 -- 172.21.15.100:0/3151468225 >> 172.21.15.100:0/3151468225 conn(0x7fb6780cdee0 msgr2=0x7fb6780ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 -- 172.21.15.100:0/3151468225 shutdown_connections 2024-08-17T02:22:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.864+0000 7fb677fff700 1 -- 172.21.15.100:0/3151468225 wait complete. 2024-08-17T02:22:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb677fff700 1 Processor -- start 2024-08-17T02:22:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb677fff700 1 -- start start 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb677fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6780c2c80 0x7fb6780a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb677fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb677fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb678140910 0x7fb67815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb677fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780a5990 con 0x7fb6780cbab0 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb677fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67815b920 con 0x7fb678140910 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb677fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67813fe00 con 0x7fb6780c2c80 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50038/0 (socket says 172.21.15.100:50038) 2024-08-17T02:22:35.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 -- 172.21.15.100:0/1012045912 learned_addr learned my addr 172.21.15.100:0/1012045912 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:35.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb676ffd700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6780c2c80 0x7fb6780a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6777fe700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb678140910 0x7fb67815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 -- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6780c2c80 msgr2=0x7fb6780a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6780c2c80 0x7fb6780a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 -- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb678140910 msgr2=0x7fb67815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:35.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb678140910 0x7fb67815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:35.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 -- 172.21.15.100:0/1012045912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6680094f0 con 0x7fb6780cbab0 2024-08-17T02:22:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.868+0000 7fb6767fc700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780af750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb67806fbe0 tx=0x7fb668017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb66effd700 1 -- 172.21.15.100:0/1012045912 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668011e00 con 0x7fb6780cbab0 2024-08-17T02:22:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb66effd700 1 -- 172.21.15.100:0/1012045912 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb668026450 con 0x7fb6780cbab0 2024-08-17T02:22:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6780afc90 con 0x7fb6780cbab0 2024-08-17T02:22:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb66effd700 1 -- 172.21.15.100:0/1012045912 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668026810 con 0x7fb6780cbab0 2024-08-17T02:22:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67815aff0 con 0x7fb6780cbab0 2024-08-17T02:22:35.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6780638f0 con 0x7fb6780cbab0 2024-08-17T02:22:35.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb66effd700 1 -- 172.21.15.100:0/1012045912 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb668026c40 con 0x7fb6780cbab0 2024-08-17T02:22:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb66effd700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb658060860 0x7fb658062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb66effd700 1 -- 172.21.15.100:0/1012045912 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb66807e7d0 con 0x7fb6780cbab0 2024-08-17T02:22:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.872+0000 7fb676ffd700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb658060860 0x7fb658062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.876+0000 7fb676ffd700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb658060860 0x7fb658062d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb678159c20 tx=0x7fb660008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:35.880+0000 7fb66effd700 1 -- 172.21.15.100:0/1012045912 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb66804eca0 con 0x7fb6780cbab0 2024-08-17T02:22:36.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.044+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6781400f0 con 0x7fb6780cbab0 2024-08-17T02:22:36.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.044+0000 7fb66effd700 1 -- 172.21.15.100:0/1012045912 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6680150a0 con 0x7fb6780cbab0 2024-08-17T02:22:36.048 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:36.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb658060860 msgr2=0x7fb658062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb658060860 0x7fb658062d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb678159c20 tx=0x7fb660008040 comp rx=0 tx=0).stop 2024-08-17T02:22:36.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 msgr2=0x7fb6780af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780af750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb67806fbe0 tx=0x7fb668017000 comp rx=0 tx=0).stop 2024-08-17T02:22:36.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 shutdown_connections 2024-08-17T02:22:36.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb658060860 0x7fb658062d10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb678140910 0x7fb67815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6780cbab0 0x7fb6780af750 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 --2- 172.21.15.100:0/1012045912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6780c2c80 0x7fb6780a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 >> 172.21.15.100:0/1012045912 conn(0x7fb6780cdee0 msgr2=0x7fb6780c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:36.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 shutdown_connections 2024-08-17T02:22:36.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.048+0000 7fb677fff700 1 -- 172.21.15.100:0/1012045912 wait complete. 2024-08-17T02:22:36.062 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:22:36.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.172+0000 7f293a074700 1 Processor -- start 2024-08-17T02:22:36.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.172+0000 7f293a074700 1 -- start start 2024-08-17T02:22:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2934159300 0x7f29340b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f29341596d0 0x7f2934066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 0x7f2934067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29340b2b60 con 0x7f29341596d0 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29340cba90 con 0x7f2934159300 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29340cb160 con 0x7f293415dbc0 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f29341596d0 0x7f2934066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f29341596d0 0x7f2934066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50052/0 (socket says 172.21.15.100:50052) 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 -- 172.21.15.100:0/3076823991 learned_addr learned my addr 172.21.15.100:0/3076823991 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2933fff700 1 --2- 172.21.15.100:0/3076823991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 0x7f2934067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f29337fe700 1 --2- 172.21.15.100:0/3076823991 >> v2:172.21.15.123:3300/0 conn(0x7f2934159300 0x7f29340b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 -- 172.21.15.100:0/3076823991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 msgr2=0x7f2934067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 --2- 172.21.15.100:0/3076823991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 0x7f2934067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 -- 172.21.15.100:0/3076823991 >> v2:172.21.15.123:3300/0 conn(0x7f2934159300 msgr2=0x7f29340b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 --2- 172.21.15.100:0/3076823991 >> v2:172.21.15.123:3300/0 conn(0x7f2934159300 0x7f29340b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 -- 172.21.15.100:0/3076823991 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29340cb440 con 0x7f29341596d0 2024-08-17T02:22:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f29337fe700 1 --2- 172.21.15.100:0/3076823991 >> v2:172.21.15.123:3300/0 conn(0x7f2934159300 0x7f29340b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f2932ffd700 1 --2- 172.21.15.100:0/3076823991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29341596d0 0x7f2934066e00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2934089460 tx=0x7f2924009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f816a3c94c5f455c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f29327fc700 1 -- 172.21.15.100:0/3076823991 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292400b860 con 0x7f29341596d0 2024-08-17T02:22:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f29327fc700 1 -- 172.21.15.100:0/3076823991 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292400b9c0 con 0x7f29341596d0 2024-08-17T02:22:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f29327fc700 1 -- 172.21.15.100:0/3076823991 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292400bd60 con 0x7f29341596d0 2024-08-17T02:22:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 -- 172.21.15.100:0/3076823991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29341596d0 msgr2=0x7f2934066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 --2- 172.21.15.100:0/3076823991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29341596d0 0x7f2934066e00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2934089460 tx=0x7f2924009250 comp rx=0 tx=0).stop 2024-08-17T02:22:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 -- 172.21.15.100:0/3076823991 shutdown_connections 2024-08-17T02:22:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 --2- 172.21.15.100:0/3076823991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 0x7f2934067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 --2- 172.21.15.100:0/3076823991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29341596d0 0x7f2934066e00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 --2- 172.21.15.100:0/3076823991 >> v2:172.21.15.123:3300/0 conn(0x7f2934159300 0x7f29340b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 -- 172.21.15.100:0/3076823991 >> 172.21.15.100:0/3076823991 conn(0x7f29340682c0 msgr2=0x7f29340686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 -- 172.21.15.100:0/3076823991 shutdown_connections 2024-08-17T02:22:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.176+0000 7f293a074700 1 -- 172.21.15.100:0/3076823991 wait complete. 2024-08-17T02:22:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f293a074700 1 Processor -- start 2024-08-17T02:22:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f293a074700 1 -- start start 2024-08-17T02:22:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f293a074700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2934159300 0x7f2934152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f293a074700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29341596d0 0x7f29340c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f293a074700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 0x7f29340c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f293a074700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340c8230 con 0x7f2934159300 2024-08-17T02:22:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f293a074700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340c84d0 con 0x7f29341596d0 2024-08-17T02:22:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f293a074700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29340c87b0 con 0x7f293415dbc0 2024-08-17T02:22:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2934159300 0x7f2934152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f2932ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29341596d0 0x7f29340c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2934159300 0x7f2934152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50066/0 (socket says 172.21.15.100:50066) 2024-08-17T02:22:36.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f2933fff700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 0x7f29340c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 -- 172.21.15.100:0/1075695594 learned_addr learned my addr 172.21.15.100:0/1075695594 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 -- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 msgr2=0x7f29340c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 0x7f29340c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 -- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29341596d0 msgr2=0x7f29340c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29341596d0 0x7f29340c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 -- 172.21.15.100:0/1075695594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240094f0 con 0x7f2934159300 2024-08-17T02:22:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f2932ffd700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29341596d0 0x7f29340c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f29337fe700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2934159300 0x7f2934152600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f292800c850 tx=0x7f2928008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.180+0000 7f291b7fe700 1 -- 172.21.15.100:0/1075695594 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928011600 con 0x7f2934159300 2024-08-17T02:22:36.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.184+0000 7f291b7fe700 1 -- 172.21.15.100:0/1075695594 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2928011760 con 0x7f2934159300 2024-08-17T02:22:36.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.184+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29340c8a90 con 0x7f2934159300 2024-08-17T02:22:36.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.184+0000 7f291b7fe700 1 -- 172.21.15.100:0/1075695594 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928011b00 con 0x7f2934159300 2024-08-17T02:22:36.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.184+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29340c9050 con 0x7f2934159300 2024-08-17T02:22:36.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.184+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29340645c0 con 0x7f2934159300 2024-08-17T02:22:36.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.184+0000 7f291b7fe700 1 -- 172.21.15.100:0/1075695594 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f29280193f0 con 0x7f2934159300 2024-08-17T02:22:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.188+0000 7f291b7fe700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2920060860 0x7f2920062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.188+0000 7f291b7fe700 1 -- 172.21.15.100:0/1075695594 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f292807cdc0 con 0x7f2934159300 2024-08-17T02:22:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.188+0000 7f2932ffd700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2920060860 0x7f2920062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.188+0000 7f2932ffd700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2920060860 0x7f2920062d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2934089460 tx=0x7f2924017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.192+0000 7f291b7fe700 1 -- 172.21.15.100:0/1075695594 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2928050ca0 con 0x7f2934159300 2024-08-17T02:22:36.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.356+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29340b0cf0 con 0x7f2934159300 2024-08-17T02:22:36.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f291b7fe700 1 -- 172.21.15.100:0/1075695594 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2928050830 con 0x7f2934159300 2024-08-17T02:22:36.361 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2920060860 msgr2=0x7f2920062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2920060860 0x7f2920062d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2934089460 tx=0x7f2924017000 comp rx=0 tx=0).stop 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2934159300 msgr2=0x7f2934152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2934159300 0x7f2934152600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f292800c850 tx=0x7f2928008040 comp rx=0 tx=0).stop 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 shutdown_connections 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2920060860 0x7f2920062d10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f293415dbc0 0x7f29340c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29341596d0 0x7f29340c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 --2- 172.21.15.100:0/1075695594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2934159300 0x7f2934152600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 >> 172.21.15.100:0/1075695594 conn(0x7f29340682c0 msgr2=0x7f29340cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:36.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 shutdown_connections 2024-08-17T02:22:36.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.360+0000 7f293a074700 1 -- 172.21.15.100:0/1075695594 wait complete. 2024-08-17T02:22:36.375 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:22:36.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a5725700 1 Processor -- start 2024-08-17T02:22:36.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a5725700 1 -- start start 2024-08-17T02:22:36.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a5725700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a01593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a5725700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a00cf670 con 0x7f74a00cfb00 2024-08-17T02:22:36.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a5725700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a00cea60 con 0x7f74a0067030 2024-08-17T02:22:36.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a5725700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a0159910 con 0x7f74a0067400 2024-08-17T02:22:36.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f749f7fe700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f74a00cfb00 0x7f74a0158cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57876/0 (socket says 172.21.15.100:57876) 2024-08-17T02:22:36.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f749f7fe700 1 -- 172.21.15.100:0/896580620 learned_addr learned my addr 172.21.15.100:0/896580620 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:36.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a4f24700 1 --2- 172.21.15.100:0/896580620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a01593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f749effd700 1 -- 172.21.15.100:0/896580620 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673763193 0 0) 0x7f74a00cf670 con 0x7f74a00cfb00 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f749effd700 1 -- 172.21.15.100:0/896580620 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f748c003570 con 0x7f74a00cfb00 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f749effd700 1 -- 172.21.15.100:0/896580620 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170320422 0 0) 0x7f74a00cea60 con 0x7f74a0067030 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a4f24700 1 -- 172.21.15.100:0/896580620 >> v1:172.21.15.123:6789/0 conn(0x7f74a0067030 legacy=0x7f74a00cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a4f24700 1 -- 172.21.15.100:0/896580620 >> v1:172.21.15.100:6789/0 conn(0x7f74a00cfb00 legacy=0x7f74a0158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.484+0000 7f74a4f24700 1 -- 172.21.15.100:0/896580620 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a00ced40 con 0x7f74a0067400 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a4f24700 1 --2- 172.21.15.100:0/896580620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a01593d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7498009e90 tx=0x7f7498009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf48ae5bfc3e4bfe server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f749effd700 1 -- 172.21.15.100:0/896580620 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749800e070 con 0x7f74a0067400 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f749effd700 1 -- 172.21.15.100:0/896580620 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749800b920 con 0x7f74a0067400 2024-08-17T02:22:36.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f749effd700 1 -- 172.21.15.100:0/896580620 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490001470 con 0x7f74a0067400 2024-08-17T02:22:36.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 -- 172.21.15.100:0/896580620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 msgr2=0x7f74a01593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 --2- 172.21.15.100:0/896580620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a01593d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7498009e90 tx=0x7f7498009780 comp rx=0 tx=0).stop 2024-08-17T02:22:36.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 -- 172.21.15.100:0/896580620 shutdown_connections 2024-08-17T02:22:36.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 --2- 172.21.15.100:0/896580620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a01593d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 -- 172.21.15.100:0/896580620 >> 172.21.15.100:0/896580620 conn(0x7f74a00d0b50 msgr2=0x7f74a00ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:36.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 -- 172.21.15.100:0/896580620 shutdown_connections 2024-08-17T02:22:36.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 -- 172.21.15.100:0/896580620 wait complete. 2024-08-17T02:22:36.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 Processor -- start 2024-08-17T02:22:36.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.488+0000 7f74a5725700 1 -- start start 2024-08-17T02:22:36.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a0067030 0x7f74a015c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a0147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a00cfb00 0x7f74a00b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00b4160 con 0x7f74a00cfb00 2024-08-17T02:22:36.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00b4440 con 0x7f74a0067030 2024-08-17T02:22:36.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00b4720 con 0x7f74a0067400 2024-08-17T02:22:36.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a00cfb00 0x7f74a00b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a00cfb00 0x7f74a00b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50078/0 (socket says 172.21.15.100:50078) 2024-08-17T02:22:36.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 -- 172.21.15.100:0/1077870664 learned_addr learned my addr 172.21.15.100:0/1077870664 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:36.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f749ffff700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a0067030 0x7f74a015c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f749f7fe700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a0147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 -- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 msgr2=0x7f74a0147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a0147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 -- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a0067030 msgr2=0x7f74a015c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a0067030 0x7f74a015c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 -- 172.21.15.100:0/1077870664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498009a20 con 0x7f74a00cfb00 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a4f24700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a00cfb00 0x7f74a00b5200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74980097c0 tx=0x7f7498017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f7497fff700 1 -- 172.21.15.100:0/1077870664 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498012cc0 con 0x7f74a00cfb00 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f7497fff700 1 -- 172.21.15.100:0/1077870664 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7498012e40 con 0x7f74a00cfb00 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a00b4a00 con 0x7f74a00cfb00 2024-08-17T02:22:36.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f7497fff700 1 -- 172.21.15.100:0/1077870664 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749801e6a0 con 0x7f74a00cfb00 2024-08-17T02:22:36.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a015c8e0 con 0x7f74a00cfb00 2024-08-17T02:22:36.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a00638f0 con 0x7f74a00cfb00 2024-08-17T02:22:36.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.492+0000 7f7497fff700 1 -- 172.21.15.100:0/1077870664 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f749801ead0 con 0x7f74a00cfb00 2024-08-17T02:22:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.496+0000 7f7497fff700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7480060890 0x7f7480062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.496+0000 7f7497fff700 1 -- 172.21.15.100:0/1077870664 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f749807f030 con 0x7f74a00cfb00 2024-08-17T02:22:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.496+0000 7f749ffff700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7480060890 0x7f7480062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.496+0000 7f749ffff700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7480060890 0x7f7480062d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7488007fd0 tx=0x7f7488007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.500+0000 7f7497fff700 1 -- 172.21.15.100:0/1077870664 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f749804f480 con 0x7f74a00cfb00 2024-08-17T02:22:36.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.664+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74a00ccf50 con 0x7f74a00cfb00 2024-08-17T02:22:36.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.664+0000 7f7497fff700 1 -- 172.21.15.100:0/1077870664 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7498052aa0 con 0x7f74a00cfb00 2024-08-17T02:22:36.667 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:22:36.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7480060890 msgr2=0x7f7480062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7480060890 0x7f7480062d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7488007fd0 tx=0x7f7488007d30 comp rx=0 tx=0).stop 2024-08-17T02:22:36.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a00cfb00 msgr2=0x7f74a00b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a00cfb00 0x7f74a00b5200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74980097c0 tx=0x7f7498017000 comp rx=0 tx=0).stop 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 shutdown_connections 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7480060890 0x7f7480062d40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a00cfb00 0x7f74a00b5200 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74a0067400 0x7f74a0147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 --2- 172.21.15.100:0/1077870664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a0067030 0x7f74a015c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 >> 172.21.15.100:0/1077870664 conn(0x7f74a00d0b50 msgr2=0x7f74a00c5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 shutdown_connections 2024-08-17T02:22:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.668+0000 7f74a5725700 1 -- 172.21.15.100:0/1077870664 wait complete. 2024-08-17T02:22:36.753 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2024-08-17T02:22:36.753 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:22:36.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338e987700 1 Processor -- start 2024-08-17T02:22:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338e987700 1 -- start start 2024-08-17T02:22:36.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338e987700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f338815d540 0x7f33880af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338e987700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f33880a9430 0x7f33880b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338e987700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33880b0270 con 0x7f338806e830 2024-08-17T02:22:36.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338e987700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33880af660 con 0x7f33880a9430 2024-08-17T02:22:36.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338e987700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33880af940 con 0x7f338815d540 2024-08-17T02:22:36.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338cf24700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f33880a9430 0x7f33880b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338cf24700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f33880a9430 0x7f33880b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50094/0 (socket says 172.21.15.100:50094) 2024-08-17T02:22:36.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338cf24700 1 -- 172.21.15.100:0/225735606 learned_addr learned my addr 172.21.15.100:0/225735606 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:36.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f3387fff700 1 --2- 172.21.15.100:0/225735606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f338815d540 0x7f33880af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338cf24700 1 -- 172.21.15.100:0/225735606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f338815d540 msgr2=0x7f33880af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338cf24700 1 --2- 172.21.15.100:0/225735606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f338815d540 0x7f33880af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338cf24700 1 -- 172.21.15.100:0/225735606 >> v1:172.21.15.123:6789/0 conn(0x7f338806e830 legacy=0x7f33880a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338cf24700 1 -- 172.21.15.100:0/225735606 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33880afc20 con 0x7f33880a9430 2024-08-17T02:22:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f338cf24700 1 --2- 172.21.15.100:0/225735606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33880a9430 0x7f33880b0700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f33780099b0 tx=0x7f3378009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7abbafe9859f54 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f3386ffd700 1 -- 172.21.15.100:0/225735606 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378009ea0 con 0x7f33880a9430 2024-08-17T02:22:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f3386ffd700 1 -- 172.21.15.100:0/225735606 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337800b860 con 0x7f33880a9430 2024-08-17T02:22:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.868+0000 7f3386ffd700 1 -- 172.21.15.100:0/225735606 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337800bc00 con 0x7f33880a9430 2024-08-17T02:22:36.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 -- 172.21.15.100:0/225735606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33880a9430 msgr2=0x7f33880b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 --2- 172.21.15.100:0/225735606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33880a9430 0x7f33880b0700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f33780099b0 tx=0x7f3378009250 comp rx=0 tx=0).stop 2024-08-17T02:22:36.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 -- 172.21.15.100:0/225735606 shutdown_connections 2024-08-17T02:22:36.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 --2- 172.21.15.100:0/225735606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33880a9430 0x7f33880b0700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 --2- 172.21.15.100:0/225735606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f338815d540 0x7f33880af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 -- 172.21.15.100:0/225735606 >> 172.21.15.100:0/225735606 conn(0x7f3388066e00 msgr2=0x7f3388067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:36.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 -- 172.21.15.100:0/225735606 shutdown_connections 2024-08-17T02:22:36.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 -- 172.21.15.100:0/225735606 wait complete. 2024-08-17T02:22:36.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 Processor -- start 2024-08-17T02:22:36.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 -- start start 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f338806e830 0x7f338816e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33880a9430 0x7f3388151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338815d540 0x7f338815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.872+0000 7f338e987700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338816d990 con 0x7f338815d540 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f338e987700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338816df10 con 0x7f338806e830 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f338e987700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338815e2f0 con 0x7f33880a9430 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f338cf24700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338815d540 0x7f338815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f3387fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f338806e830 0x7f338816e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f33877fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33880a9430 0x7f3388151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f338cf24700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338815d540 0x7f338815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50106/0 (socket says 172.21.15.100:50106) 2024-08-17T02:22:36.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f3387fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f338806e830 0x7f338816e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:41504/0 (socket says 172.21.15.100:41504) 2024-08-17T02:22:36.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f338cf24700 1 -- 172.21.15.100:0/1651176866 learned_addr learned my addr 172.21.15.100:0/1651176866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:36.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f33877fe700 1 -- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f338806e830 msgr2=0x7f338816e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f33877fe700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f338806e830 0x7f338816e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f33877fe700 1 -- 172.21.15.100:0/1651176866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338815d540 msgr2=0x7f338815d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:36.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f33877fe700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338815d540 0x7f338815d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:36.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f33877fe700 1 -- 172.21.15.100:0/1651176866 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33780094f0 con 0x7f33880a9430 2024-08-17T02:22:36.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f33877fe700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33880a9430 0x7f3388151d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f338000c910 tx=0x7f33800016f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f337ffff700 1 -- 172.21.15.100:0/1651176866 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380012630 con 0x7f33880a9430 2024-08-17T02:22:36.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f337ffff700 1 -- 172.21.15.100:0/1651176866 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33800127b0 con 0x7f33880a9430 2024-08-17T02:22:36.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338815e5d0 con 0x7f33880a9430 2024-08-17T02:22:36.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f337ffff700 1 -- 172.21.15.100:0/1651176866 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380012b70 con 0x7f33880a9430 2024-08-17T02:22:36.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338816e8e0 con 0x7f33880a9430 2024-08-17T02:22:36.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.876+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338815e810 con 0x7f33880a9430 2024-08-17T02:22:36.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.880+0000 7f337ffff700 1 -- 172.21.15.100:0/1651176866 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3380003030 con 0x7f33880a9430 2024-08-17T02:22:36.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.880+0000 7f337ffff700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f33680608e0 0x7f3368062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:36.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.880+0000 7f337ffff700 1 -- 172.21.15.100:0/1651176866 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f338007cb00 con 0x7f33880a9430 2024-08-17T02:22:36.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.880+0000 7f3387fff700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f33680608e0 0x7f3368062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:36.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.880+0000 7f3387fff700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f33680608e0 0x7f3368062d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3370009d10 tx=0x7f33700091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:36.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:36.888+0000 7f337ffff700 1 -- 172.21.15.100:0/1651176866 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f338815e810 con 0x7f33880a9430 2024-08-17T02:22:37.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.052+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f338816e080 con 0x7f33880a9430 2024-08-17T02:22:37.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.056+0000 7f337ffff700 1 -- 172.21.15.100:0/1651176866 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f338004ceb0 con 0x7f33880a9430 2024-08-17T02:22:37.059 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:37.059 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f33680608e0 msgr2=0x7f3368062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f33680608e0 0x7f3368062d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3370009d10 tx=0x7f33700091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33880a9430 msgr2=0x7f3388151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33880a9430 0x7f3388151d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f338000c910 tx=0x7f33800016f0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 shutdown_connections 2024-08-17T02:22:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f33680608e0 0x7f3368062d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338815d540 0x7f338815d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33880a9430 0x7f3388151d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 --2- 172.21.15.100:0/1651176866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f338806e830 0x7f338816e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 >> 172.21.15.100:0/1651176866 conn(0x7f3388066e00 msgr2=0x7f338815c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 shutdown_connections 2024-08-17T02:22:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.060+0000 7f338e987700 1 -- 172.21.15.100:0/1651176866 wait complete. 2024-08-17T02:22:37.079 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:02.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:03.458625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:22:37.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:37.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.192+0000 7f753a577700 1 Processor -- start 2024-08-17T02:22:37.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.192+0000 7f753a577700 1 -- start start 2024-08-17T02:22:37.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7534067510 0x7f75340ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7534159480 0x7f75340cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75340ce210 con 0x7f75341590b0 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75340cba00 con 0x7f7534067510 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75340cdc90 con 0x7f7534159480 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7538d74700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7534067510 0x7f75340ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7538d74700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7534067510 0x7f75340ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50116/0 (socket says 172.21.15.100:50116) 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7538d74700 1 -- 172.21.15.100:0/2451803393 learned_addr learned my addr 172.21.15.100:0/2451803393 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7539d76700 1 --2- 172.21.15.100:0/2451803393 >> v2:172.21.15.123:3300/0 conn(0x7f7534159480 0x7f75340cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7538d74700 1 -- 172.21.15.100:0/2451803393 >> v2:172.21.15.123:3300/0 conn(0x7f7534159480 msgr2=0x7f75340cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7538d74700 1 --2- 172.21.15.100:0/2451803393 >> v2:172.21.15.123:3300/0 conn(0x7f7534159480 0x7f75340cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7538d74700 1 -- 172.21.15.100:0/2451803393 >> v1:172.21.15.123:6789/0 conn(0x7f75341590b0 legacy=0x7f7534066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7538d74700 1 -- 172.21.15.100:0/2451803393 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75340cb440 con 0x7f7534067510 2024-08-17T02:22:37.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f7538d74700 1 --2- 172.21.15.100:0/2451803393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7534067510 0x7f75340ce6a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7534071060 tx=0x7f7524009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203cb0305fb4223d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f752bfff700 1 -- 172.21.15.100:0/2451803393 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400b820 con 0x7f7534067510 2024-08-17T02:22:37.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f752bfff700 1 -- 172.21.15.100:0/2451803393 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400b980 con 0x7f7534067510 2024-08-17T02:22:37.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f752bfff700 1 -- 172.21.15.100:0/2451803393 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400bd20 con 0x7f7534067510 2024-08-17T02:22:37.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 -- 172.21.15.100:0/2451803393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7534067510 msgr2=0x7f75340ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 --2- 172.21.15.100:0/2451803393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7534067510 0x7f75340ce6a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7534071060 tx=0x7f7524009250 comp rx=0 tx=0).stop 2024-08-17T02:22:37.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 -- 172.21.15.100:0/2451803393 shutdown_connections 2024-08-17T02:22:37.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 --2- 172.21.15.100:0/2451803393 >> v2:172.21.15.123:3300/0 conn(0x7f7534159480 0x7f75340cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 --2- 172.21.15.100:0/2451803393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7534067510 0x7f75340ce6a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 -- 172.21.15.100:0/2451803393 >> 172.21.15.100:0/2451803393 conn(0x7f75340d0b90 msgr2=0x7f75340ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:37.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 -- 172.21.15.100:0/2451803393 shutdown_connections 2024-08-17T02:22:37.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.196+0000 7f753a577700 1 -- 172.21.15.100:0/2451803393 wait complete. 2024-08-17T02:22:37.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f753a577700 1 Processor -- start 2024-08-17T02:22:37.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f753a577700 1 -- start start 2024-08-17T02:22:37.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f753a577700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7534067510 0x7f7534152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f753a577700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75341590b0 0x7f753415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f753a577700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7534159480 0x7f75340c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f753a577700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753415bc40 con 0x7f75341590b0 2024-08-17T02:22:37.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f753a577700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7534152fe0 con 0x7f7534159480 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f753a577700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753415c780 con 0x7f7534067510 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75341590b0 0x7f753415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75341590b0 0x7f753415e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50124/0 (socket says 172.21.15.100:50124) 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 -- 172.21.15.100:0/3234010360 learned_addr learned my addr 172.21.15.100:0/3234010360 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7539d76700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7534159480 0x7f75340c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7539575700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7534067510 0x7f7534152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 -- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7534067510 msgr2=0x7f7534152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7534067510 0x7f7534152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 -- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7534159480 msgr2=0x7f75340c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7534159480 0x7f75340c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 -- 172.21.15.100:0/3234010360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75240094f0 con 0x7f75341590b0 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7539575700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7534067510 0x7f7534152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:37.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.200+0000 7f7538d74700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75341590b0 0x7f753415e370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f75240030c0 tx=0x7f7524016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f75297fa700 1 -- 172.21.15.100:0/3234010360 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524011e60 con 0x7f75341590b0 2024-08-17T02:22:37.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f75297fa700 1 -- 172.21.15.100:0/3234010360 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7524020070 con 0x7f75341590b0 2024-08-17T02:22:37.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f75297fa700 1 -- 172.21.15.100:0/3234010360 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752401b6a0 con 0x7f75341590b0 2024-08-17T02:22:37.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753415be50 con 0x7f75341590b0 2024-08-17T02:22:37.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753415c410 con 0x7f75341590b0 2024-08-17T02:22:37.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7534064330 con 0x7f75341590b0 2024-08-17T02:22:37.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f75297fa700 1 -- 172.21.15.100:0/3234010360 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7524022030 con 0x7f75341590b0 2024-08-17T02:22:37.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f75297fa700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7518060890 0x7f7518062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f75297fa700 1 -- 172.21.15.100:0/3234010360 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f752407ec70 con 0x7f75341590b0 2024-08-17T02:22:37.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f7539575700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7518060890 0x7f7518062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.204+0000 7f7539575700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7518060890 0x7f7518062d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7530002880 tx=0x7f7530009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.212+0000 7f75297fa700 1 -- 172.21.15.100:0/3234010360 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f752404f060 con 0x7f75341590b0 2024-08-17T02:22:37.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.372+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7534064330 con 0x7f75341590b0 2024-08-17T02:22:37.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.376+0000 7f75297fa700 1 -- 172.21.15.100:0/3234010360 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7524052680 con 0x7f75341590b0 2024-08-17T02:22:37.377 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:37.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.376+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7518060890 msgr2=0x7f7518062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.376+0000 7f753a577700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7518060890 0x7f7518062d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7530002880 tx=0x7f7530009040 comp rx=0 tx=0).stop 2024-08-17T02:22:37.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.376+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75341590b0 msgr2=0x7f753415e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.376+0000 7f753a577700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75341590b0 0x7f753415e370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f75240030c0 tx=0x7f7524016000 comp rx=0 tx=0).stop 2024-08-17T02:22:37.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.376+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 shutdown_connections 2024-08-17T02:22:37.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.376+0000 7f753a577700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7518060890 0x7f7518062d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.380+0000 7f753a577700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7534159480 0x7f75340c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.380+0000 7f753a577700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75341590b0 0x7f753415e370 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.380+0000 7f753a577700 1 --2- 172.21.15.100:0/3234010360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7534067510 0x7f7534152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.380+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 >> 172.21.15.100:0/3234010360 conn(0x7f75340d0b90 msgr2=0x7f75340c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:37.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.380+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 shutdown_connections 2024-08-17T02:22:37.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.380+0000 7f753a577700 1 -- 172.21.15.100:0/3234010360 wait complete. 2024-08-17T02:22:37.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:22:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd6898e700 1 Processor -- start 2024-08-17T02:22:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd6898e700 1 -- start start 2024-08-17T02:22:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd6898e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd64142660 0x7efd64142a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd6898e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd640c82e0 0x7efd640c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd6898e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd640b1180 con 0x7efd640ae510 2024-08-17T02:22:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd6898e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640c9310 con 0x7efd640c82e0 2024-08-17T02:22:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd6898e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640b0890 con 0x7efd64142660 2024-08-17T02:22:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd63fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd640c82e0 0x7efd640c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd63fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd640c82e0 0x7efd640c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50134/0 (socket says 172.21.15.100:50134) 2024-08-17T02:22:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd63fff700 1 -- 172.21.15.100:0/3912058396 learned_addr learned my addr 172.21.15.100:0/3912058396 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd637fe700 1 --2- 172.21.15.100:0/3912058396 >> v2:172.21.15.123:3300/0 conn(0x7efd64142660 0x7efd64142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd63fff700 1 -- 172.21.15.100:0/3912058396 >> v2:172.21.15.123:3300/0 conn(0x7efd64142660 msgr2=0x7efd64142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd63fff700 1 --2- 172.21.15.100:0/3912058396 >> v2:172.21.15.123:3300/0 conn(0x7efd64142660 0x7efd64142a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd63fff700 1 -- 172.21.15.100:0/3912058396 >> v1:172.21.15.123:6789/0 conn(0x7efd640ae510 legacy=0x7efd640ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd63fff700 1 -- 172.21.15.100:0/3912058396 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640c9030 con 0x7efd640c82e0 2024-08-17T02:22:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.508+0000 7efd63fff700 1 --2- 172.21.15.100:0/3912058396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640c82e0 0x7efd640c8760 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd6406d2e0 tx=0x7efd5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ea2b74e1f08492 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd627fc700 1 -- 172.21.15.100:0/3912058396 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00e070 con 0x7efd640c82e0 2024-08-17T02:22:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd627fc700 1 -- 172.21.15.100:0/3912058396 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c00ba50 con 0x7efd640c82e0 2024-08-17T02:22:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd627fc700 1 -- 172.21.15.100:0/3912058396 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0092e0 con 0x7efd640c82e0 2024-08-17T02:22:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 -- 172.21.15.100:0/3912058396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640c82e0 msgr2=0x7efd640c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 --2- 172.21.15.100:0/3912058396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640c82e0 0x7efd640c8760 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd6406d2e0 tx=0x7efd5c009840 comp rx=0 tx=0).stop 2024-08-17T02:22:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 -- 172.21.15.100:0/3912058396 shutdown_connections 2024-08-17T02:22:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 --2- 172.21.15.100:0/3912058396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640c82e0 0x7efd640c8760 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 --2- 172.21.15.100:0/3912058396 >> v2:172.21.15.123:3300/0 conn(0x7efd64142660 0x7efd64142a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 -- 172.21.15.100:0/3912058396 >> 172.21.15.100:0/3912058396 conn(0x7efd640c9830 msgr2=0x7efd640c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 -- 172.21.15.100:0/3912058396 shutdown_connections 2024-08-17T02:22:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 -- 172.21.15.100:0/3912058396 wait complete. 2024-08-17T02:22:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 Processor -- start 2024-08-17T02:22:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.512+0000 7efd6898e700 1 -- start start 2024-08-17T02:22:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd6898e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640ae510 0x7efd6415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd6898e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd640c82e0 0x7efd641580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd6898e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd64142660 0x7efd64158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd6898e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64158b60 con 0x7efd640ae510 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd6898e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd641335e0 con 0x7efd640c82e0 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd6898e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd641338c0 con 0x7efd64142660 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640ae510 0x7efd6415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640ae510 0x7efd6415ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50150/0 (socket says 172.21.15.100:50150) 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 -- 172.21.15.100:0/1525835048 learned_addr learned my addr 172.21.15.100:0/1525835048 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd63fff700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd64142660 0x7efd64158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd62ffd700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd640c82e0 0x7efd641580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 -- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd64142660 msgr2=0x7efd64158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd64142660 0x7efd64158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 -- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd640c82e0 msgr2=0x7efd641580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd640c82e0 0x7efd641580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 -- 172.21.15.100:0/1525835048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c009ae0 con 0x7efd640ae510 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd637fe700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640ae510 0x7efd6415ca70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd5400d890 tx=0x7efd54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd4b7fe700 1 -- 172.21.15.100:0/1525835048 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400b600 con 0x7efd640ae510 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd4b7fe700 1 -- 172.21.15.100:0/1525835048 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5400b760 con 0x7efd640ae510 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd4b7fe700 1 -- 172.21.15.100:0/1525835048 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400bae0 con 0x7efd640ae510 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd64133ba0 con 0x7efd640ae510 2024-08-17T02:22:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.516+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd641341e0 con 0x7efd640ae510 2024-08-17T02:22:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.520+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd640638f0 con 0x7efd640ae510 2024-08-17T02:22:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.520+0000 7efd4b7fe700 1 -- 172.21.15.100:0/1525835048 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efd5401e460 con 0x7efd640ae510 2024-08-17T02:22:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.520+0000 7efd4b7fe700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd50060890 0x7efd50062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.520+0000 7efd4b7fe700 1 -- 172.21.15.100:0/1525835048 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd5407d290 con 0x7efd640ae510 2024-08-17T02:22:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.520+0000 7efd62ffd700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd50060890 0x7efd50062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.520+0000 7efd62ffd700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd50060890 0x7efd50062d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efd640c6cf0 tx=0x7efd58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.524+0000 7efd4b7fe700 1 -- 172.21.15.100:0/1525835048 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7efd5404d6e0 con 0x7efd640ae510 2024-08-17T02:22:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd640ca5c0 con 0x7efd640ae510 2024-08-17T02:22:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd4b7fe700 1 -- 172.21.15.100:0/1525835048 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd54050d00 con 0x7efd640ae510 2024-08-17T02:22:37.690 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:22:37.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd50060890 msgr2=0x7efd50062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd50060890 0x7efd50062d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efd640c6cf0 tx=0x7efd58009040 comp rx=0 tx=0).stop 2024-08-17T02:22:37.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640ae510 msgr2=0x7efd6415ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640ae510 0x7efd6415ca70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd5400d890 tx=0x7efd54008040 comp rx=0 tx=0).stop 2024-08-17T02:22:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 shutdown_connections 2024-08-17T02:22:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efd50060890 0x7efd50062d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd64142660 0x7efd64158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd640c82e0 0x7efd641580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 --2- 172.21.15.100:0/1525835048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd640ae510 0x7efd6415ca70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 >> 172.21.15.100:0/1525835048 conn(0x7efd640c9830 msgr2=0x7efd64159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 shutdown_connections 2024-08-17T02:22:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.688+0000 7efd6898e700 1 -- 172.21.15.100:0/1525835048 wait complete. 2024-08-17T02:22:37.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:22:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.816+0000 7f4b520e3700 1 Processor -- start 2024-08-17T02:22:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.816+0000 7f4b520e3700 1 -- start start 2024-08-17T02:22:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b4c1505b0 0x7f4b4c0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b4c068be0 0x7f4b4c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4c157160 con 0x7f4b4c1501e0 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c069020 con 0x7f4b4c1505b0 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c156550 con 0x7f4b4c068be0 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b508e0700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b4c1505b0 0x7f4b4c0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b508e0700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b4c1505b0 0x7f4b4c0686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50156/0 (socket says 172.21.15.100:50156) 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b508e0700 1 -- 172.21.15.100:0/3329934117 learned_addr learned my addr 172.21.15.100:0/3329934117 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b518e2700 1 --2- 172.21.15.100:0/3329934117 >> v2:172.21.15.123:3300/0 conn(0x7f4b4c068be0 0x7f4b4c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b508e0700 1 -- 172.21.15.100:0/3329934117 >> v2:172.21.15.123:3300/0 conn(0x7f4b4c068be0 msgr2=0x7f4b4c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b508e0700 1 --2- 172.21.15.100:0/3329934117 >> v2:172.21.15.123:3300/0 conn(0x7f4b4c068be0 0x7f4b4c0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b508e0700 1 -- 172.21.15.100:0/3329934117 >> v1:172.21.15.123:6789/0 conn(0x7f4b4c1501e0 legacy=0x7f4b4c0ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b508e0700 1 -- 172.21.15.100:0/3329934117 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c156830 con 0x7f4b4c1505b0 2024-08-17T02:22:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b508e0700 1 --2- 172.21.15.100:0/3329934117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1505b0 0x7f4b4c0686a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0b1c60 tx=0x7f4b40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ef30f4e935522c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b3ffff700 1 -- 172.21.15.100:0/3329934117 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000b7e0 con 0x7f4b4c1505b0 2024-08-17T02:22:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b3ffff700 1 -- 172.21.15.100:0/3329934117 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4000b940 con 0x7f4b4c1505b0 2024-08-17T02:22:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b3ffff700 1 -- 172.21.15.100:0/3329934117 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000bce0 con 0x7f4b4c1505b0 2024-08-17T02:22:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 -- 172.21.15.100:0/3329934117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1505b0 msgr2=0x7f4b4c0686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 --2- 172.21.15.100:0/3329934117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1505b0 0x7f4b4c0686a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0b1c60 tx=0x7f4b40009250 comp rx=0 tx=0).stop 2024-08-17T02:22:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 -- 172.21.15.100:0/3329934117 shutdown_connections 2024-08-17T02:22:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 --2- 172.21.15.100:0/3329934117 >> v2:172.21.15.123:3300/0 conn(0x7f4b4c068be0 0x7f4b4c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 --2- 172.21.15.100:0/3329934117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1505b0 0x7f4b4c0686a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 -- 172.21.15.100:0/3329934117 >> 172.21.15.100:0/3329934117 conn(0x7f4b4c06e310 msgr2=0x7f4b4c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 -- 172.21.15.100:0/3329934117 shutdown_connections 2024-08-17T02:22:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.820+0000 7f4b520e3700 1 -- 172.21.15.100:0/3329934117 wait complete. 2024-08-17T02:22:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b520e3700 1 Processor -- start 2024-08-17T02:22:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b520e3700 1 -- start start 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b520e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c068be0 0x7f4b4c13d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b520e3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1501e0 0x7f4b4c148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b520e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c1505b0 0x7f4b4c14a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b520e3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c1467f0 con 0x7f4b4c1501e0 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b520e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c13d560 con 0x7f4b4c1505b0 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b520e3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c145be0 con 0x7f4b4c068be0 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b508e0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1501e0 0x7f4b4c148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b508e0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1501e0 0x7f4b4c148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50162/0 (socket says 172.21.15.100:50162) 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b508e0700 1 -- 172.21.15.100:0/616569446 learned_addr learned my addr 172.21.15.100:0/616569446 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b510e1700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c068be0 0x7f4b4c13d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.824+0000 7f4b518e2700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c1505b0 0x7f4b4c14a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b510e1700 1 -- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c1505b0 msgr2=0x7f4b4c14a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b510e1700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c1505b0 0x7f4b4c14a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b510e1700 1 -- 172.21.15.100:0/616569446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1501e0 msgr2=0x7f4b4c148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b510e1700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1501e0 0x7f4b4c148900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b510e1700 1 -- 172.21.15.100:0/616569446 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400094f0 con 0x7f4b4c068be0 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b518e2700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c1505b0 0x7f4b4c14a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b508e0700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1501e0 0x7f4b4c148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b510e1700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c068be0 0x7f4b4c13d020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800c4e0 tx=0x7f4b48007ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b3d7fa700 1 -- 172.21.15.100:0/616569446 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800ce90 con 0x7f4b4c068be0 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b3d7fa700 1 -- 172.21.15.100:0/616569446 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b48011590 con 0x7f4b4c068be0 2024-08-17T02:22:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c145ec0 con 0x7f4b4c068be0 2024-08-17T02:22:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b3d7fa700 1 -- 172.21.15.100:0/616569446 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48011930 con 0x7f4b4c068be0 2024-08-17T02:22:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c146480 con 0x7f4b4c068be0 2024-08-17T02:22:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4c0638f0 con 0x7f4b4c068be0 2024-08-17T02:22:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.828+0000 7f4b3d7fa700 1 -- 172.21.15.100:0/616569446 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4b4801c460 con 0x7f4b4c068be0 2024-08-17T02:22:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.832+0000 7f4b3d7fa700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b300608e0 0x7f4b30062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.832+0000 7f4b3d7fa700 1 -- 172.21.15.100:0/616569446 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4b4807d350 con 0x7f4b4c068be0 2024-08-17T02:22:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.832+0000 7f4b508e0700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b300608e0 0x7f4b30062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.832+0000 7f4b508e0700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b300608e0 0x7f4b30062d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009fd0 tx=0x7f4b40016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:37.840+0000 7f4b3d7fa700 1 -- 172.21.15.100:0/616569446 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4b4804d780 con 0x7f4b4c068be0 2024-08-17T02:22:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.004+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b4c155630 con 0x7f4b4c068be0 2024-08-17T02:22:38.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.004+0000 7f4b3d7fa700 1 -- 172.21.15.100:0/616569446 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b48050da0 con 0x7f4b4c068be0 2024-08-17T02:22:38.009 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:22:38.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.008+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b300608e0 msgr2=0x7f4b30062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.008+0000 7f4b520e3700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b300608e0 0x7f4b30062d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009fd0 tx=0x7f4b40016000 comp rx=0 tx=0).stop 2024-08-17T02:22:38.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.008+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c068be0 msgr2=0x7f4b4c13d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.008+0000 7f4b520e3700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c068be0 0x7f4b4c13d020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800c4e0 tx=0x7f4b48007ea0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.012+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 shutdown_connections 2024-08-17T02:22:38.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.012+0000 7f4b520e3700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b300608e0 0x7f4b30062d90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.012+0000 7f4b520e3700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c1505b0 0x7f4b4c14a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.012+0000 7f4b520e3700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c1501e0 0x7f4b4c148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.012+0000 7f4b520e3700 1 --2- 172.21.15.100:0/616569446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c068be0 0x7f4b4c13d020 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.012+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 >> 172.21.15.100:0/616569446 conn(0x7f4b4c06e310 msgr2=0x7f4b4c0ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:38.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.012+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 shutdown_connections 2024-08-17T02:22:38.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.012+0000 7f4b520e3700 1 -- 172.21.15.100:0/616569446 wait complete. 2024-08-17T02:22:38.027 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-17T02:22:38.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa742dd2700 1 Processor -- start 2024-08-17T02:22:38.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa742dd2700 1 -- start start 2024-08-17T02:22:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa742dd2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c159040 0x7fa73c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa742dd2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa73c0d0010 0x7fa73c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa742dd2700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa73c15cea0 0x7fa73c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa742dd2700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0cf190 con 0x7fa73c15cea0 2024-08-17T02:22:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa742dd2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0d09a0 con 0x7fa73c0d0010 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa742dd2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0ce5e0 con 0x7fa73c159040 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.132+0000 7fa7425d1700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa73c15cea0 0x7fa73c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7425d1700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa73c15cea0 0x7fa73c15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50176/0 (socket says 172.21.15.100:50176) 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7425d1700 1 -- 172.21.15.100:0/1924996114 learned_addr learned my addr 172.21.15.100:0/1924996114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7415cf700 1 --2- 172.21.15.100:0/1924996114 >> v2:172.21.15.123:3300/0 conn(0x7fa73c0d0010 0x7fa73c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa741dd0700 1 --2- 172.21.15.100:0/1924996114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c159040 0x7fa73c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7425d1700 1 -- 172.21.15.100:0/1924996114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c159040 msgr2=0x7fa73c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7425d1700 1 --2- 172.21.15.100:0/1924996114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c159040 0x7fa73c0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7425d1700 1 -- 172.21.15.100:0/1924996114 >> v2:172.21.15.123:3300/0 conn(0x7fa73c0d0010 msgr2=0x7fa73c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7425d1700 1 --2- 172.21.15.100:0/1924996114 >> v2:172.21.15.123:3300/0 conn(0x7fa73c0d0010 0x7fa73c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7425d1700 1 -- 172.21.15.100:0/1924996114 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0ce860 con 0x7fa73c15cea0 2024-08-17T02:22:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa7425d1700 1 --2- 172.21.15.100:0/1924996114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c15cea0 0x7fa73c15d320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa73c067f20 tx=0x7fa734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2cbe816a6caaab server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa740dce700 1 -- 172.21.15.100:0/1924996114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734009e70 con 0x7fa73c15cea0 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa740dce700 1 -- 172.21.15.100:0/1924996114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73400b860 con 0x7fa73c15cea0 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa740dce700 1 -- 172.21.15.100:0/1924996114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73400bc00 con 0x7fa73c15cea0 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 -- 172.21.15.100:0/1924996114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c15cea0 msgr2=0x7fa73c15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1924996114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c15cea0 0x7fa73c15d320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa73c067f20 tx=0x7fa734009250 comp rx=0 tx=0).stop 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 -- 172.21.15.100:0/1924996114 shutdown_connections 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1924996114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c15cea0 0x7fa73c15d320 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1924996114 >> v2:172.21.15.123:3300/0 conn(0x7fa73c0d0010 0x7fa73c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1924996114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c159040 0x7fa73c0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 -- 172.21.15.100:0/1924996114 >> 172.21.15.100:0/1924996114 conn(0x7fa73c068250 msgr2=0x7fa73c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 -- 172.21.15.100:0/1924996114 shutdown_connections 2024-08-17T02:22:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.136+0000 7fa742dd2700 1 -- 172.21.15.100:0/1924996114 wait complete. 2024-08-17T02:22:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa742dd2700 1 Processor -- start 2024-08-17T02:22:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa742dd2700 1 -- start start 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa742dd2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa73c0d0010 0x7fa73c0b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa742dd2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c159040 0x7fa73c0634a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa742dd2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c15cea0 0x7fa73c151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa742dd2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c16cd50 con 0x7fa73c159040 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa742dd2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c16cff0 con 0x7fa73c0d0010 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa742dd2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c16d2d0 con 0x7fa73c15cea0 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa7415cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c159040 0x7fa73c0634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa7415cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c159040 0x7fa73c0634a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50190/0 (socket says 172.21.15.100:50190) 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.140+0000 7fa7415cf700 1 -- 172.21.15.100:0/1356132421 learned_addr learned my addr 172.21.15.100:0/1356132421 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:38.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa741dd0700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa73c0d0010 0x7fa73c0b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa7425d1700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c15cea0 0x7fa73c151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa7415cf700 1 -- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c15cea0 msgr2=0x7fa73c151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa7415cf700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c15cea0 0x7fa73c151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa7415cf700 1 -- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa73c0d0010 msgr2=0x7fa73c0b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa7415cf700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa73c0d0010 0x7fa73c0b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa7415cf700 1 -- 172.21.15.100:0/1356132421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340094f0 con 0x7fa73c159040 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa7425d1700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c15cea0 0x7fa73c151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa7415cf700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c159040 0x7fa73c0634a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa738009ef0 tx=0x7fa73800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa731ffb700 1 -- 172.21.15.100:0/1356132421 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73800cc20 con 0x7fa73c159040 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa731ffb700 1 -- 172.21.15.100:0/1356132421 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73800f040 con 0x7fa73c159040 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa731ffb700 1 -- 172.21.15.100:0/1356132421 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738014480 con 0x7fa73c159040 2024-08-17T02:22:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c16d5b0 con 0x7fa73c159040 2024-08-17T02:22:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c169710 con 0x7fa73c159040 2024-08-17T02:22:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa731ffb700 1 -- 172.21.15.100:0/1356132421 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa7380147f0 con 0x7fa73c159040 2024-08-17T02:22:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c0ccea0 con 0x7fa73c159040 2024-08-17T02:22:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa731ffb700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa728060890 0x7fa728062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa731ffb700 1 -- 172.21.15.100:0/1356132421 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa73807d520 con 0x7fa73c159040 2024-08-17T02:22:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.144+0000 7fa741dd0700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa728060890 0x7fa728062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.148+0000 7fa741dd0700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa728060890 0x7fa728062d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa72c006fd0 tx=0x7fa72c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:38.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.152+0000 7fa731ffb700 1 -- 172.21.15.100:0/1356132421 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa73804d950 con 0x7fa73c159040 2024-08-17T02:22:38.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.376+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fa73c0cfdf0 con 0x7fa73c159040 2024-08-17T02:22:38.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.396+0000 7fa731ffb700 1 -- 172.21.15.100:0/1356132421 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa738050f70 con 0x7fa73c159040 2024-08-17T02:22:38.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa728060890 msgr2=0x7fa728062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa728060890 0x7fa728062d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa72c006fd0 tx=0x7fa72c008040 comp rx=0 tx=0).stop 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c159040 msgr2=0x7fa73c0634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c159040 0x7fa73c0634a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa738009ef0 tx=0x7fa73800b040 comp rx=0 tx=0).stop 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 shutdown_connections 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fa728060890 0x7fa728062d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa73c15cea0 0x7fa73c151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73c159040 0x7fa73c0634a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 --2- 172.21.15.100:0/1356132421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa73c0d0010 0x7fa73c0b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 >> 172.21.15.100:0/1356132421 conn(0x7fa73c068250 msgr2=0x7fa73c0aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 shutdown_connections 2024-08-17T02:22:38.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.400+0000 7fa742dd2700 1 -- 172.21.15.100:0/1356132421 wait complete. 2024-08-17T02:22:38.419 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-17T02:22:38.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f25008f3700 1 Processor -- start 2024-08-17T02:22:38.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f25008f3700 1 -- start start 2024-08-17T02:22:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f25008f3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24fc1501b0 0x7f24fc150580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f25008f3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24fc0686a0 0x7f24fc068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f25008f3700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24fc069040 con 0x7f24fc0ab860 2024-08-17T02:22:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f25008f3700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24fc15a3b0 con 0x7f24fc0686a0 2024-08-17T02:22:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f25008f3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24fc0b0550 con 0x7f24fc1501b0 2024-08-17T02:22:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f24fa59c700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f24fc0ab860 0x7f24fc156e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57884/0 (socket says 172.21.15.100:57884) 2024-08-17T02:22:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f24fa59c700 1 -- 172.21.15.100:0/1711263974 learned_addr learned my addr 172.21.15.100:0/1711263974 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f24fb59e700 1 --2- 172.21.15.100:0/1711263974 >> v2:172.21.15.100:3300/0 conn(0x7f24fc0686a0 0x7f24fc068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.532+0000 7f24fad9d700 1 --2- 172.21.15.100:0/1711263974 >> v2:172.21.15.123:3300/0 conn(0x7f24fc1501b0 0x7f24fc150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24f9d9b700 1 -- 172.21.15.100:0/1711263974 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781366539 0 0) 0x7f24f0001110 con 0x7f24fc0ab860 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24f9d9b700 1 -- 172.21.15.100:0/1711263974 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24e4003570 con 0x7f24fc0ab860 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24fb59e700 1 -- 172.21.15.100:0/1711263974 >> v2:172.21.15.123:3300/0 conn(0x7f24fc1501b0 msgr2=0x7f24fc150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24fb59e700 1 --2- 172.21.15.100:0/1711263974 >> v2:172.21.15.123:3300/0 conn(0x7f24fc1501b0 0x7f24fc150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24fb59e700 1 -- 172.21.15.100:0/1711263974 >> v1:172.21.15.100:6789/0 conn(0x7f24fc0ab860 legacy=0x7f24fc156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24fb59e700 1 -- 172.21.15.100:0/1711263974 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc159ac0 con 0x7f24fc0686a0 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24fb59e700 1 --2- 172.21.15.100:0/1711263974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0686a0 0x7f24fc068b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f24fc069310 tx=0x7f24f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657754dce4ed0d8f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24f9d9b700 1 -- 172.21.15.100:0/1711263974 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f400bab0 con 0x7f24fc0686a0 2024-08-17T02:22:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24f9d9b700 1 -- 172.21.15.100:0/1711263974 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f400bc10 con 0x7f24fc0686a0 2024-08-17T02:22:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f24f9d9b700 1 -- 172.21.15.100:0/1711263974 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f40104d0 con 0x7f24fc0686a0 2024-08-17T02:22:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f25008f3700 1 -- 172.21.15.100:0/1711263974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0686a0 msgr2=0x7f24fc068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f25008f3700 1 --2- 172.21.15.100:0/1711263974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0686a0 0x7f24fc068b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f24fc069310 tx=0x7f24f4009780 comp rx=0 tx=0).stop 2024-08-17T02:22:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f25008f3700 1 -- 172.21.15.100:0/1711263974 shutdown_connections 2024-08-17T02:22:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f25008f3700 1 --2- 172.21.15.100:0/1711263974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0686a0 0x7f24fc068b00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f25008f3700 1 --2- 172.21.15.100:0/1711263974 >> v2:172.21.15.123:3300/0 conn(0x7f24fc1501b0 0x7f24fc150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f25008f3700 1 -- 172.21.15.100:0/1711263974 >> 172.21.15.100:0/1711263974 conn(0x7f24fc06e350 msgr2=0x7f24fc06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f25008f3700 1 -- 172.21.15.100:0/1711263974 shutdown_connections 2024-08-17T02:22:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.536+0000 7f25008f3700 1 -- 172.21.15.100:0/1711263974 wait complete. 2024-08-17T02:22:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f25008f3700 1 Processor -- start 2024-08-17T02:22:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f25008f3700 1 -- start start 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f25008f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24fc0686a0 0x7f24fc144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f25008f3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0ab860 0x7f24fc14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f25008f3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc1501b0 0x7f24fc14f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f25008f3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc144760 con 0x7f24fc0ab860 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f25008f3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc153280 con 0x7f24fc1501b0 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f25008f3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1523f0 con 0x7f24fc0686a0 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0ab860 0x7f24fc14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0ab860 0x7f24fc14f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50202/0 (socket says 172.21.15.100:50202) 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 -- 172.21.15.100:0/1424390006 learned_addr learned my addr 172.21.15.100:0/1424390006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fad9d700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24fc0686a0 0x7f24fc144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fb59e700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc1501b0 0x7f24fc14f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 -- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24fc0686a0 msgr2=0x7f24fc144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24fc0686a0 0x7f24fc144220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 -- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc1501b0 msgr2=0x7f24fc14f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc1501b0 0x7f24fc14f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 -- 172.21.15.100:0/1424390006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4009a20 con 0x7f24fc0ab860 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.540+0000 7f24fa59c700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0ab860 0x7f24fc14f060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c7c0 tx=0x7f24f0007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.544+0000 7f24e2ffd700 1 -- 172.21.15.100:0/1424390006 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0005c00 con 0x7f24fc0ab860 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.544+0000 7f24e2ffd700 1 -- 172.21.15.100:0/1424390006 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f0005d60 con 0x7f24fc0ab860 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.544+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc152670 con 0x7f24fc0ab860 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.544+0000 7f24e2ffd700 1 -- 172.21.15.100:0/1424390006 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0007320 con 0x7f24fc0ab860 2024-08-17T02:22:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.544+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc152c30 con 0x7f24fc0ab860 2024-08-17T02:22:38.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.544+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc0638f0 con 0x7f24fc0ab860 2024-08-17T02:22:38.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.544+0000 7f24e2ffd700 1 -- 172.21.15.100:0/1424390006 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f24f001b3b0 con 0x7f24fc0ab860 2024-08-17T02:22:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.548+0000 7f24e2ffd700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24e8060890 0x7f24e8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.548+0000 7f24e2ffd700 1 -- 172.21.15.100:0/1424390006 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f24f007c020 con 0x7f24fc0ab860 2024-08-17T02:22:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.548+0000 7f24fad9d700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24e8060890 0x7f24e8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.548+0000 7f24fad9d700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24e8060890 0x7f24e8062d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009d10 tx=0x7f24ec0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:38.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.556+0000 7f24e2ffd700 1 -- 172.21.15.100:0/1424390006 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f24f004c3d0 con 0x7f24fc0ab860 2024-08-17T02:22:38.708 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:22:38.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:22:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.720+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f24fc069040 con 0x7f24fc0ab860 2024-08-17T02:22:38.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f24e2ffd700 1 -- 172.21.15.100:0/1424390006 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f24fc069040 con 0x7f24fc0ab860 2024-08-17T02:22:38.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24e8060890 msgr2=0x7f24e8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24e8060890 0x7f24e8062d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009d10 tx=0x7f24ec0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0ab860 msgr2=0x7f24fc14f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0ab860 0x7f24fc14f060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c7c0 tx=0x7f24f0007c70 comp rx=0 tx=0).stop 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 shutdown_connections 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24e8060890 0x7f24e8062d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc1501b0 0x7f24fc14f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc0ab860 0x7f24fc14f060 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 --2- 172.21.15.100:0/1424390006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24fc0686a0 0x7f24fc144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 >> 172.21.15.100:0/1424390006 conn(0x7f24fc06e350 msgr2=0x7f24fc14c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 shutdown_connections 2024-08-17T02:22:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:38.728+0000 7f25008f3700 1 -- 172.21.15.100:0/1424390006 wait complete. 2024-08-17T02:22:38.744 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-17T02:22:38.744 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-17T02:22:38.758 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-17T02:22:38.779 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-17T02:22:38.779 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-08-17T02:22:38.779 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:22:38.779 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:38.798 INFO:teuthology.orchestra.run:waiting for 900 2024-08-17T02:22:38.798 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:22:38.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:38.798 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-17T02:22:38.852 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-08-17T02:22:38.853 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:22:38.853 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-08-17T02:22:38.880 INFO:teuthology.orchestra.run:waiting for 900 2024-08-17T02:22:38.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:22:38.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:38.881 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-17T02:22:38.934 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-08-17T02:22:38.934 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:22:38.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-08-17T02:22:38.955 INFO:teuthology.orchestra.run:waiting for 900 2024-08-17T02:22:38.955 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:22:38.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:38.955 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-17T02:22:38.963 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-08-17T02:22:38.963 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:22:38.963 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-08-17T02:22:38.979 INFO:teuthology.orchestra.run:waiting for 900 2024-08-17T02:22:38.980 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:22:38.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:38.980 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-17T02:22:39.030 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-08-17T02:22:39.031 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:39.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 Processor -- start 2024-08-17T02:22:39.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 -- start start 2024-08-17T02:22:39.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4c015d540 0x7ff4c00af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c00a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c00b0270 con 0x7ff4c00a9260 2024-08-17T02:22:39.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c00af660 con 0x7ff4c015d540 2024-08-17T02:22:39.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00af940 con 0x7ff4c006e830 2024-08-17T02:22:39.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c00a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4c015d540 0x7ff4c00af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c00a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:49144/0 (socket says 172.21.15.100:49144) 2024-08-17T02:22:39.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4c015d540 0x7ff4c00af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:41636/0 (socket says 172.21.15.100:41636) 2024-08-17T02:22:39.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bf7fe700 1 -- 172.21.15.100:0/359213101 learned_addr learned my addr 172.21.15.100:0/359213101 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:39.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c4c46700 1 --1- 172.21.15.100:0/359213101 >> v1:172.21.15.123:6789/0 conn(0x7ff4c00a9260 0x7ff4c00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:58298/0 (socket says 172.21.15.100:58298) 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bffff700 1 -- 172.21.15.100:0/359213101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 msgr2=0x7ff4c00a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bffff700 1 --2- 172.21.15.100:0/359213101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c00a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bffff700 1 -- 172.21.15.100:0/359213101 >> v1:172.21.15.123:6789/0 conn(0x7ff4c00a9260 legacy=0x7ff4c00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bffff700 1 -- 172.21.15.100:0/359213101 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c00afc20 con 0x7ff4c015d540 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bf7fe700 1 --2- 172.21.15.100:0/359213101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c00a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4bffff700 1 --2- 172.21.15.100:0/359213101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c015d540 0x7ff4c00af030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80098e0 tx=0x7ff4a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618c30f0ef92e352 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4beffd700 1 -- 172.21.15.100:0/359213101 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800b640 con 0x7ff4c015d540 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4beffd700 1 -- 172.21.15.100:0/359213101 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a800b7a0 con 0x7ff4c015d540 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4beffd700 1 -- 172.21.15.100:0/359213101 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800bb40 con 0x7ff4c015d540 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 -- 172.21.15.100:0/359213101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c015d540 msgr2=0x7ff4c00af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.144+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/359213101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c015d540 0x7ff4c00af030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80098e0 tx=0x7ff4a80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 -- 172.21.15.100:0/359213101 shutdown_connections 2024-08-17T02:22:39.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/359213101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c00a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/359213101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c015d540 0x7ff4c00af030 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 -- 172.21.15.100:0/359213101 >> 172.21.15.100:0/359213101 conn(0x7ff4c0066e00 msgr2=0x7ff4c0067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:39.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 -- 172.21.15.100:0/359213101 shutdown_connections 2024-08-17T02:22:39.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 -- 172.21.15.100:0/359213101 wait complete. 2024-08-17T02:22:39.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 Processor -- start 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 -- start start 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c01716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c00a9260 0x7ff4c00ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c015d540 0x7ff4c00969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c0099b20 con 0x7ff4c015d540 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c0098f10 con 0x7ff4c00a9260 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c66a9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00991f0 con 0x7ff4c006e830 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c00a9260 0x7ff4c00ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c4c46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c015d540 0x7ff4c00969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c00a9260 0x7ff4c00ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:41652/0 (socket says 172.21.15.100:41652) 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c4c46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c015d540 0x7ff4c00969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50208/0 (socket says 172.21.15.100:50208) 2024-08-17T02:22:39.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4bf7fe700 1 -- 172.21.15.100:0/873105374 learned_addr learned my addr 172.21.15.100:0/873105374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:39.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4bffff700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c01716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c4c46700 1 -- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 msgr2=0x7ff4c01716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c4c46700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c01716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c4c46700 1 -- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c00a9260 msgr2=0x7ff4c00ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.148+0000 7ff4c4c46700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c00a9260 0x7ff4c00ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4c4c46700 1 -- 172.21.15.100:0/873105374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8009ae0 con 0x7ff4c015d540 2024-08-17T02:22:39.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4bf7fe700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c00a9260 0x7ff4c00ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:39.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4c4c46700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c015d540 0x7ff4c00969f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c4e0 tx=0x7ff4b0007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4b7fff700 1 -- 172.21.15.100:0/873105374 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b000ce90 con 0x7ff4c015d540 2024-08-17T02:22:39.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4b7fff700 1 -- 172.21.15.100:0/873105374 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b0005a70 con 0x7ff4c015d540 2024-08-17T02:22:39.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a8009420 con 0x7ff4c015d540 2024-08-17T02:22:39.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4b7fff700 1 -- 172.21.15.100:0/873105374 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0005e30 con 0x7ff4c015d540 2024-08-17T02:22:39.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c00997b0 con 0x7ff4c015d540 2024-08-17T02:22:39.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c0099c90 con 0x7ff4c015d540 2024-08-17T02:22:39.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.152+0000 7ff4b7fff700 1 -- 172.21.15.100:0/873105374 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff4b00072e0 con 0x7ff4c015d540 2024-08-17T02:22:39.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.156+0000 7ff4b7fff700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff4a0060860 0x7ff4a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.156+0000 7ff4b7fff700 1 -- 172.21.15.100:0/873105374 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4b007bf00 con 0x7ff4c015d540 2024-08-17T02:22:39.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.156+0000 7ff4bffff700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff4a0060860 0x7ff4a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.156+0000 7ff4bffff700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff4a0060860 0x7ff4a0062d10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80093f0 tx=0x7ff4a8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.164+0000 7ff4b7fff700 1 -- 172.21.15.100:0/873105374 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff4b004c2b0 con 0x7ff4c015d540 2024-08-17T02:22:39.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.368+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4c00645c0 con 0x7ff4c015d540 2024-08-17T02:22:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.372+0000 7ff4b7fff700 1 -- 172.21.15.100:0/873105374 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4603 (secure 0 0 0) 0x7ff4b004f8d0 con 0x7ff4c015d540 2024-08-17T02:22:39.374 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:39.374 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:04.459540+0000","modified":"2024-08-17T02:22:07.136998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:22:39.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff4a0060860 msgr2=0x7ff4a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff4a0060860 0x7ff4a0062d10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80093f0 tx=0x7ff4a8017000 comp rx=0 tx=0).stop 2024-08-17T02:22:39.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c015d540 msgr2=0x7ff4c00969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c015d540 0x7ff4c00969f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c4e0 tx=0x7ff4b0007e30 comp rx=0 tx=0).stop 2024-08-17T02:22:39.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 shutdown_connections 2024-08-17T02:22:39.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff4a0060860 0x7ff4a0062d10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c015d540 0x7ff4c00969f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c00a9260 0x7ff4c00ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 --2- 172.21.15.100:0/873105374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c006e830 0x7ff4c01716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 >> 172.21.15.100:0/873105374 conn(0x7ff4c0066e00 msgr2=0x7ff4c0158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:39.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 shutdown_connections 2024-08-17T02:22:39.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.376+0000 7ff4c66a9700 1 -- 172.21.15.100:0/873105374 wait complete. 2024-08-17T02:22:39.379 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-08-17T02:22:39.393 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.492+0000 7fbe892ab700 1 Processor -- start 2024-08-17T02:22:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.492+0000 7fbe892ab700 1 -- start start 2024-08-17T02:22:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbe8413fa70 0x7fbe840af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe840af570 0x7fbe840b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe840ad730 con 0x7fbe8413f6a0 2024-08-17T02:22:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840acb20 con 0x7fbe8413fa70 2024-08-17T02:22:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840ace00 con 0x7fbe840af570 2024-08-17T02:22:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe83fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbe8413f6a0 0x7fbe840adbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57886/0 (socket says 172.21.15.100:57886) 2024-08-17T02:22:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe83fff700 1 -- 172.21.15.100:0/1335478298 learned_addr learned my addr 172.21.15.100:0/1335478298 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe88aaa700 1 --2- 172.21.15.100:0/1335478298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe840af570 0x7fbe840b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe837fe700 1 --2- 172.21.15.100:0/1335478298 >> v2:172.21.15.123:3300/0 conn(0x7fbe8413fa70 0x7fbe840af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe82ffd700 1 -- 172.21.15.100:0/1335478298 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137554486 0 0) 0x7fbe840ad730 con 0x7fbe8413f6a0 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe82ffd700 1 -- 172.21.15.100:0/1335478298 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe6c003570 con 0x7fbe8413f6a0 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe837fe700 1 -- 172.21.15.100:0/1335478298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe840af570 msgr2=0x7fbe840b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe837fe700 1 --2- 172.21.15.100:0/1335478298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe840af570 0x7fbe840b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe837fe700 1 -- 172.21.15.100:0/1335478298 >> v1:172.21.15.100:6789/0 conn(0x7fbe8413f6a0 legacy=0x7fbe840adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe837fe700 1 -- 172.21.15.100:0/1335478298 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840ad0e0 con 0x7fbe8413fa70 2024-08-17T02:22:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe837fe700 1 --2- 172.21.15.100:0/1335478298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8413fa70 0x7fbe840af030 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe780098e0 tx=0x7fbe780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0f8fd6fd1d57db server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe82ffd700 1 -- 172.21.15.100:0/1335478298 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78011630 con 0x7fbe8413fa70 2024-08-17T02:22:39.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe82ffd700 1 -- 172.21.15.100:0/1335478298 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78011790 con 0x7fbe8413fa70 2024-08-17T02:22:39.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe82ffd700 1 -- 172.21.15.100:0/1335478298 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78011b30 con 0x7fbe8413fa70 2024-08-17T02:22:39.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 -- 172.21.15.100:0/1335478298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8413fa70 msgr2=0x7fbe840af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1335478298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8413fa70 0x7fbe840af030 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe780098e0 tx=0x7fbe780091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 -- 172.21.15.100:0/1335478298 shutdown_connections 2024-08-17T02:22:39.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1335478298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe840af570 0x7fbe840b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1335478298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8413fa70 0x7fbe840af030 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 -- 172.21.15.100:0/1335478298 >> 172.21.15.100:0/1335478298 conn(0x7fbe8406eae0 msgr2=0x7fbe84067f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:39.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 -- 172.21.15.100:0/1335478298 shutdown_connections 2024-08-17T02:22:39.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.496+0000 7fbe892ab700 1 -- 172.21.15.100:0/1335478298 wait complete. 2024-08-17T02:22:39.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 Processor -- start 2024-08-17T02:22:39.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 -- start start 2024-08-17T02:22:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe840af570 0x7fbe84169fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8413f6a0 0x7fbe84154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe8413fa70 0x7fbe840aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840a9b50 con 0x7fbe8413fa70 2024-08-17T02:22:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840a9da0 con 0x7fbe840af570 2024-08-17T02:22:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840aa080 con 0x7fbe8413f6a0 2024-08-17T02:22:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe8413fa70 0x7fbe840aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe837fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8413f6a0 0x7fbe84154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe8413fa70 0x7fbe840aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50224/0 (socket says 172.21.15.100:50224) 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 -- 172.21.15.100:0/1765006963 learned_addr learned my addr 172.21.15.100:0/1765006963 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe83fff700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe840af570 0x7fbe84169fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 -- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8413f6a0 msgr2=0x7fbe84154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8413f6a0 0x7fbe84154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 -- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe840af570 msgr2=0x7fbe84169fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe840af570 0x7fbe84169fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 -- 172.21.15.100:0/1765006963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78009420 con 0x7fbe8413fa70 2024-08-17T02:22:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe88aaa700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe8413fa70 0x7fbe840aba70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009d10 tx=0x7fbe7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe6bfff700 1 -- 172.21.15.100:0/1765006963 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c009e70 con 0x7fbe8413fa70 2024-08-17T02:22:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe6bfff700 1 -- 172.21.15.100:0/1765006963 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c00f040 con 0x7fbe8413fa70 2024-08-17T02:22:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe840aa360 con 0x7fbe8413fa70 2024-08-17T02:22:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe6bfff700 1 -- 172.21.15.100:0/1765006963 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00ccf0 con 0x7fbe8413fa70 2024-08-17T02:22:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.500+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe84154990 con 0x7fbe8413fa70 2024-08-17T02:22:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.504+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe840643a0 con 0x7fbe8413fa70 2024-08-17T02:22:39.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.504+0000 7fbe6bfff700 1 -- 172.21.15.100:0/1765006963 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbe7c014690 con 0x7fbe8413fa70 2024-08-17T02:22:39.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.504+0000 7fbe6bfff700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe70060860 0x7fbe70062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.504+0000 7fbe6bfff700 1 -- 172.21.15.100:0/1765006963 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbe7c07d690 con 0x7fbe8413fa70 2024-08-17T02:22:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.504+0000 7fbe83fff700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe70060860 0x7fbe70062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.504+0000 7fbe83fff700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe70060860 0x7fbe70062d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbe74007fd0 tx=0x7fbe74009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.508+0000 7fbe6bfff700 1 -- 172.21.15.100:0/1765006963 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbe7c0514d0 con 0x7fbe8413fa70 2024-08-17T02:22:39.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.672+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe8406f6f0 con 0x7fbe8413fa70 2024-08-17T02:22:39.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe6bfff700 1 -- 172.21.15.100:0/1765006963 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fbe7c051060 con 0x7fbe8413fa70 2024-08-17T02:22:39.678 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:39.678 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:39.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe70060860 msgr2=0x7fbe70062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe70060860 0x7fbe70062d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbe74007fd0 tx=0x7fbe74009040 comp rx=0 tx=0).stop 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe8413fa70 msgr2=0x7fbe840aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe8413fa70 0x7fbe840aba70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009d10 tx=0x7fbe7c00b040 comp rx=0 tx=0).stop 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 shutdown_connections 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbe70060860 0x7fbe70062d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe8413fa70 0x7fbe840aba70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8413f6a0 0x7fbe84154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 --2- 172.21.15.100:0/1765006963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe840af570 0x7fbe84169fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 >> 172.21.15.100:0/1765006963 conn(0x7fbe8406eae0 msgr2=0x7fbe8406b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 shutdown_connections 2024-08-17T02:22:39.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.676+0000 7fbe892ab700 1 -- 172.21.15.100:0/1765006963 wait complete. 2024-08-17T02:22:39.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:02.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:03.458625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:22:39.694 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:39.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.804+0000 7f40f5e01700 1 Processor -- start 2024-08-17T02:22:39.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.804+0000 7f40f5e01700 1 -- start start 2024-08-17T02:22:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5e01700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f40f006e470 0x7f40f00aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5e01700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 0x7f40f00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5e01700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f0140f70 con 0x7f40f00b0ef0 2024-08-17T02:22:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5e01700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f00b1a80 con 0x7f40f006e470 2024-08-17T02:22:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5e01700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0141250 con 0x7f40f006e840 2024-08-17T02:22:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5600700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f40f00b0ef0 0x7f40f00b1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57892/0 (socket says 172.21.15.100:57892) 2024-08-17T02:22:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5600700 1 -- 172.21.15.100:0/2777797621 learned_addr learned my addr 172.21.15.100:0/2777797621 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40effff700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 0x7f40f00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f4dff700 1 --2- 172.21.15.100:0/2777797621 >> v2:172.21.15.123:3300/0 conn(0x7f40f006e470 0x7f40f00aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663185831 0 0) 0x7f40f0140f70 con 0x7f40f00b0ef0 2024-08-17T02:22:39.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e0003570 con 0x7f40f00b0ef0 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2186972382 0 0) 0x7f40e0003570 con 0x7f40f00b0ef0 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 msgr2=0x7f40f00aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 0x7f40f00aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 >> v2:172.21.15.123:3300/0 conn(0x7f40f006e470 msgr2=0x7f40f00aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 --2- 172.21.15.100:0/2777797621 >> v2:172.21.15.123:3300/0 conn(0x7f40f006e470 0x7f40f00aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f0141530 con 0x7f40f00b0ef0 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f40e80017f0 con 0x7f40f00b0ef0 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 >> v1:172.21.15.100:6789/0 conn(0x7f40f00b0ef0 legacy=0x7f40f00b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e0003a60 0x7f40e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 0x7f40e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e0004d50 0x7f40e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0141250 con 0x7f40e0003a60 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e000ba60 con 0x7f40e0004d50 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e000bba0 con 0x7f40f006e840 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f4dff700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e0003a60 0x7f40e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40effff700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 0x7f40e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5600700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e0004d50 0x7f40e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40effff700 1 -- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e0004d50 msgr2=0x7f40e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40effff700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e0004d50 0x7f40e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40effff700 1 -- 172.21.15.100:0/2777797621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e0003a60 msgr2=0x7f40e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40effff700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e0003a60 0x7f40e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40effff700 1 -- 172.21.15.100:0/2777797621 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e0003570 con 0x7f40f006e840 2024-08-17T02:22:39.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40f5600700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e0004d50 0x7f40e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40effff700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 0x7f40e0004810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009c00 tx=0x7f40dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29541314f811f0c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00e6b0 con 0x7f40f006e840 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc00e810 con 0x7f40f006e840 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.808+0000 7f40ef7fe700 1 -- 172.21.15.100:0/2777797621 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00ebd0 con 0x7f40f006e840 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 msgr2=0x7f40e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 --2- 172.21.15.100:0/2777797621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f006e840 0x7f40e0004810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009c00 tx=0x7f40dc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5600700 1 -- 172.21.15.100:0/2777797621 reap_dead start 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- 172.21.15.100:0/2777797621 shutdown_connections 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- 172.21.15.100:0/2777797621 >> 172.21.15.100:0/2777797621 conn(0x7f40f00678a0 msgr2=0x7f40f0067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- 172.21.15.100:0/2777797621 shutdown_connections 2024-08-17T02:22:39.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- 172.21.15.100:0/2777797621 wait complete. 2024-08-17T02:22:39.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 Processor -- start 2024-08-17T02:22:39.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- start start 2024-08-17T02:22:39.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f0140820 0x7f40f01561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40f0156720 0x7f40f01628f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0162e30 0x7f40f016dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f01632b0 con 0x7f40f0156720 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f01595a0 con 0x7f40f0162e30 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5e01700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0159270 con 0x7f40f0140820 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40effff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40f0156720 0x7f40f01628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40effff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40f0156720 0x7f40f01628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50238/0 (socket says 172.21.15.100:50238) 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40effff700 1 -- 172.21.15.100:0/4073972242 learned_addr learned my addr 172.21.15.100:0/4073972242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f4dff700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f0140820 0x7f40f01561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40f5600700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0162e30 0x7f40f016dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40effff700 1 -- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f0140820 msgr2=0x7f40f01561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40effff700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f0140820 0x7f40f01561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40effff700 1 -- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0162e30 msgr2=0x7f40f016dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40effff700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0162e30 0x7f40f016dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.812+0000 7f40effff700 1 -- 172.21.15.100:0/4073972242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f01639a0 con 0x7f40f0156720 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40effff700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40f0156720 0x7f40f01628f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f40e000bd30 tx=0x7f40dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40ecff9700 1 -- 172.21.15.100:0/4073972242 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc022070 con 0x7f40f0156720 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40ecff9700 1 -- 172.21.15.100:0/4073972242 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc00e980 con 0x7f40f0156720 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f0163c80 con 0x7f40f0156720 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40ecff9700 1 -- 172.21.15.100:0/4073972242 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00e6b0 con 0x7f40f0156720 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f0156e10 con 0x7f40f0156720 2024-08-17T02:22:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f0163230 con 0x7f40f0156720 2024-08-17T02:22:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40ecff9700 1 -- 172.21.15.100:0/4073972242 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f40dc01c660 con 0x7f40f0156720 2024-08-17T02:22:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40ecff9700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f40d4060860 0x7f40d4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.816+0000 7f40ecff9700 1 -- 172.21.15.100:0/4073972242 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f40dc07e0f0 con 0x7f40f0156720 2024-08-17T02:22:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.820+0000 7f40f4dff700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f40d4060860 0x7f40d4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.820+0000 7f40f4dff700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f40d4060860 0x7f40d4062d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f40e4006f00 tx=0x7f40e40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:39.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.828+0000 7f40ecff9700 1 -- 172.21.15.100:0/4073972242 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f40dc052080 con 0x7f40f0156720 2024-08-17T02:22:39.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.988+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40f0064330 con 0x7f40f0156720 2024-08-17T02:22:39.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.988+0000 7f40ecff9700 1 -- 172.21.15.100:0/4073972242 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40dc051c10 con 0x7f40f0156720 2024-08-17T02:22:39.992 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f40d4060860 msgr2=0x7f40d4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f40d4060860 0x7f40d4062d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f40e4006f00 tx=0x7f40e40091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40f0156720 msgr2=0x7f40f01628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40f0156720 0x7f40f01628f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f40e000bd30 tx=0x7f40dc01a000 comp rx=0 tx=0).stop 2024-08-17T02:22:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 shutdown_connections 2024-08-17T02:22:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f40d4060860 0x7f40d4062d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0162e30 0x7f40f016dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40f0156720 0x7f40f01628f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 --2- 172.21.15.100:0/4073972242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f0140820 0x7f40f01561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:39.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 >> 172.21.15.100:0/4073972242 conn(0x7f40f00678a0 msgr2=0x7f40f015c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:39.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 shutdown_connections 2024-08-17T02:22:39.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:39.992+0000 7f40f5e01700 1 -- 172.21.15.100:0/4073972242 wait complete. 2024-08-17T02:22:40.005 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:22:40.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feae13f7700 1 Processor -- start 2024-08-17T02:22:40.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feae13f7700 1 -- start start 2024-08-17T02:22:40.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feae13f7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feadc1514d0 0x7feadc06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feae13f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc0b14a0 0x7feadc157990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feae13f7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc06e010 con 0x7feadc1574f0 2024-08-17T02:22:40.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feae13f7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc06efa0 con 0x7feadc1514d0 2024-08-17T02:22:40.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feae13f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc157fb0 con 0x7feadc0b14a0 2024-08-17T02:22:40.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadbfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feadc1514d0 0x7feadc06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feae0bf6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc0b14a0 0x7feadc157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadbfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feadc1514d0 0x7feadc06ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:41700/0 (socket says 172.21.15.100:41700) 2024-08-17T02:22:40.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadbfff700 1 -- 172.21.15.100:0/448810296 learned_addr learned my addr 172.21.15.100:0/448810296 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:40.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7fead3fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7feadc1574f0 0x7feadc0b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:58300/0 (socket says 172.21.15.100:58300) 2024-08-17T02:22:40.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadbfff700 1 -- 172.21.15.100:0/448810296 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc0b14a0 msgr2=0x7feadc157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:40.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadbfff700 1 --2- 172.21.15.100:0/448810296 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc0b14a0 0x7feadc157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadbfff700 1 -- 172.21.15.100:0/448810296 >> v1:172.21.15.123:6789/0 conn(0x7feadc1574f0 legacy=0x7feadc0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadbfff700 1 -- 172.21.15.100:0/448810296 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc158290 con 0x7feadc1514d0 2024-08-17T02:22:40.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadbfff700 1 --2- 172.21.15.100:0/448810296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc1514d0 0x7feadc06ea60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feacc0099b0 tx=0x7feacc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c858097af8a50b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadb7fe700 1 -- 172.21.15.100:0/448810296 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc009ea0 con 0x7feadc1514d0 2024-08-17T02:22:40.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadb7fe700 1 -- 172.21.15.100:0/448810296 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc00b860 con 0x7feadc1514d0 2024-08-17T02:22:40.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.116+0000 7feadb7fe700 1 -- 172.21.15.100:0/448810296 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc00bc00 con 0x7feadc1514d0 2024-08-17T02:22:40.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- 172.21.15.100:0/448810296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc1514d0 msgr2=0x7feadc06ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 --2- 172.21.15.100:0/448810296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc1514d0 0x7feadc06ea60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feacc0099b0 tx=0x7feacc009250 comp rx=0 tx=0).stop 2024-08-17T02:22:40.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- 172.21.15.100:0/448810296 shutdown_connections 2024-08-17T02:22:40.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 --2- 172.21.15.100:0/448810296 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc0b14a0 0x7feadc157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 --2- 172.21.15.100:0/448810296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc1514d0 0x7feadc06ea60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- 172.21.15.100:0/448810296 >> 172.21.15.100:0/448810296 conn(0x7feadc0678a0 msgr2=0x7feadc067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:40.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- 172.21.15.100:0/448810296 shutdown_connections 2024-08-17T02:22:40.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- 172.21.15.100:0/448810296 wait complete. 2024-08-17T02:22:40.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 Processor -- start 2024-08-17T02:22:40.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- start start 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc0b14a0 0x7feadc1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feadc1514d0 0x7feadc0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc1574f0 0x7feadc069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc152fd0 con 0x7feadc1514d0 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0b0bc0 con 0x7feadc0b14a0 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae13f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0b0ea0 con 0x7feadc1574f0 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7fead3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feadc1514d0 0x7feadc0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7fead3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feadc1514d0 0x7feadc0b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50248/0 (socket says 172.21.15.100:50248) 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7fead3fff700 1 -- 172.21.15.100:0/135971403 learned_addr learned my addr 172.21.15.100:0/135971403 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feae0bf6700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc1574f0 0x7feadc069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.120+0000 7feadbfff700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc0b14a0 0x7feadc1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead3fff700 1 -- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc1574f0 msgr2=0x7feadc069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead3fff700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc1574f0 0x7feadc069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead3fff700 1 -- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc0b14a0 msgr2=0x7feadc1508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead3fff700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc0b14a0 0x7feadc1508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead3fff700 1 -- 172.21.15.100:0/135971403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc0094f0 con 0x7feadc1514d0 2024-08-17T02:22:40.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead3fff700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feadc1514d0 0x7feadc0b0290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feac400c560 tx=0x7feac4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead8ff9700 1 -- 172.21.15.100:0/135971403 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4005a70 con 0x7feadc1514d0 2024-08-17T02:22:40.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead8ff9700 1 -- 172.21.15.100:0/135971403 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac4005bd0 con 0x7feadc1514d0 2024-08-17T02:22:40.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feadc0b1180 con 0x7feadc1514d0 2024-08-17T02:22:40.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead8ff9700 1 -- 172.21.15.100:0/135971403 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac40070d0 con 0x7feadc1514d0 2024-08-17T02:22:40.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc152ba0 con 0x7feadc1514d0 2024-08-17T02:22:40.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feadc06b1b0 con 0x7feadc1514d0 2024-08-17T02:22:40.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead8ff9700 1 -- 172.21.15.100:0/135971403 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7feac4007480 con 0x7feadc1514d0 2024-08-17T02:22:40.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.124+0000 7fead8ff9700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feabc060860 0x7feabc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.128+0000 7fead8ff9700 1 -- 172.21.15.100:0/135971403 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feac407bf10 con 0x7feadc1514d0 2024-08-17T02:22:40.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.128+0000 7feadbfff700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feabc060860 0x7feabc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.128+0000 7feadbfff700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feabc060860 0x7feabc062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feacc0099b0 tx=0x7feacc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.136+0000 7fead8ff9700 1 -- 172.21.15.100:0/135971403 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7feadc06b1b0 con 0x7feadc1514d0 2024-08-17T02:22:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feadc06b1b0 con 0x7feadc1514d0 2024-08-17T02:22:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7fead8ff9700 1 -- 172.21.15.100:0/135971403 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7feac4017020 con 0x7feadc1514d0 2024-08-17T02:22:40.298 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feabc060860 msgr2=0x7feabc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feabc060860 0x7feabc062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feacc0099b0 tx=0x7feacc017000 comp rx=0 tx=0).stop 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feadc1514d0 msgr2=0x7feadc0b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feadc1514d0 0x7feadc0b0290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feac400c560 tx=0x7feac4007f70 comp rx=0 tx=0).stop 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 shutdown_connections 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feabc060860 0x7feabc062d10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feadc1574f0 0x7feadc069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feadc1514d0 0x7feadc0b0290 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 --2- 172.21.15.100:0/135971403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc0b14a0 0x7feadc1508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.296+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 >> 172.21.15.100:0/135971403 conn(0x7feadc0678a0 msgr2=0x7feadc0c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.300+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 shutdown_connections 2024-08-17T02:22:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.300+0000 7feae13f7700 1 -- 172.21.15.100:0/135971403 wait complete. 2024-08-17T02:22:40.312 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:22:40.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f06a1b99700 1 Processor -- start 2024-08-17T02:22:40.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f06a1b99700 1 -- start start 2024-08-17T02:22:40.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f06a1b99700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f069c0a9040 0x7f069c0a9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f06a1b99700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 0x7f069c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f06a1b99700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c0aa890 con 0x7f069c0adf20 2024-08-17T02:22:40.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f06a1b99700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0aa030 con 0x7f069c0a9040 2024-08-17T02:22:40.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f06a1b99700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0ad330 con 0x7f069c0acf00 2024-08-17T02:22:40.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f069b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f069c0a9040 0x7f069c0a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f069bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f069c0adf20 0x7f069c1569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:58316/0 (socket says 172.21.15.100:58316) 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069bfff700 1 -- 172.21.15.100:0/4215923358 learned_addr learned my addr 172.21.15.100:0/4215923358 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.416+0000 7f069affd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 0x7f069c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069b7fe700 1 -- 172.21.15.100:0/4215923358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 msgr2=0x7f069c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069b7fe700 1 --2- 172.21.15.100:0/4215923358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 0x7f069c06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069b7fe700 1 -- 172.21.15.100:0/4215923358 >> v1:172.21.15.123:6789/0 conn(0x7f069c0adf20 legacy=0x7f069c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069b7fe700 1 -- 172.21.15.100:0/4215923358 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0ae730 con 0x7f069c0a9040 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069b7fe700 1 --2- 172.21.15.100:0/4215923358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0a9040 0x7f069c0a9410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0694009e60 tx=0x7f0694009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d2f08c58122485 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069a7fc700 1 -- 172.21.15.100:0/4215923358 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069400e070 con 0x7f069c0a9040 2024-08-17T02:22:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069a7fc700 1 -- 172.21.15.100:0/4215923358 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069400bb60 con 0x7f069c0a9040 2024-08-17T02:22:40.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f069a7fc700 1 -- 172.21.15.100:0/4215923358 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694012430 con 0x7f069c0a9040 2024-08-17T02:22:40.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f06a1b99700 1 -- 172.21.15.100:0/4215923358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0a9040 msgr2=0x7f069c0a9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f06a1b99700 1 --2- 172.21.15.100:0/4215923358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0a9040 0x7f069c0a9410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0694009e60 tx=0x7f0694009780 comp rx=0 tx=0).stop 2024-08-17T02:22:40.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f06a1b99700 1 -- 172.21.15.100:0/4215923358 shutdown_connections 2024-08-17T02:22:40.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f06a1b99700 1 --2- 172.21.15.100:0/4215923358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 0x7f069c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f06a1b99700 1 --2- 172.21.15.100:0/4215923358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0a9040 0x7f069c0a9410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f06a1b99700 1 -- 172.21.15.100:0/4215923358 >> 172.21.15.100:0/4215923358 conn(0x7f069c066e00 msgr2=0x7f069c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:40.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f06a1b99700 1 -- 172.21.15.100:0/4215923358 shutdown_connections 2024-08-17T02:22:40.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.420+0000 7f06a1b99700 1 -- 172.21.15.100:0/4215923358 wait complete. 2024-08-17T02:22:40.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.424+0000 7f06a1b99700 1 Processor -- start 2024-08-17T02:22:40.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.424+0000 7f06a1b99700 1 -- start start 2024-08-17T02:22:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.424+0000 7f06a1b99700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f069c0a9040 0x7f069c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.424+0000 7f06a1b99700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 0x7f069c15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.424+0000 7f06a1b99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0adf20 0x7f069c0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.424+0000 7f06a1b99700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c1587e0 con 0x7f069c0a9040 2024-08-17T02:22:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.424+0000 7f06a1b99700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c152d90 con 0x7f069c0adf20 2024-08-17T02:22:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.424+0000 7f06a1b99700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c8c30 con 0x7f069c0acf00 2024-08-17T02:22:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f069c0a9040 0x7f069c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f069c0a9040 0x7f069c152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50258/0 (socket says 172.21.15.100:50258) 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 -- 172.21.15.100:0/389706575 learned_addr learned my addr 172.21.15.100:0/389706575 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069affd700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 0x7f069c15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069bfff700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0adf20 0x7f069c0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 -- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 msgr2=0x7f069c15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 0x7f069c15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 -- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0adf20 msgr2=0x7f069c0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0adf20 0x7f069c0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 -- 172.21.15.100:0/389706575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0694009a20 con 0x7f069c0a9040 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f069b7fe700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f069c0a9040 0x7f069c152850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f069400b6c0 tx=0x7f0694019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f06837fe700 1 -- 172.21.15.100:0/389706575 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694012cf0 con 0x7f069c0a9040 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f06837fe700 1 -- 172.21.15.100:0/389706575 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694012e50 con 0x7f069c0a9040 2024-08-17T02:22:40.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c0c8f10 con 0x7f069c0a9040 2024-08-17T02:22:40.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f06837fe700 1 -- 172.21.15.100:0/389706575 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940276c0 con 0x7f069c0a9040 2024-08-17T02:22:40.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c1584c0 con 0x7f069c0a9040 2024-08-17T02:22:40.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c0645c0 con 0x7f069c0a9040 2024-08-17T02:22:40.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.428+0000 7f06837fe700 1 -- 172.21.15.100:0/389706575 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0694024070 con 0x7f069c0a9040 2024-08-17T02:22:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.432+0000 7f06837fe700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0688060860 0x7f0688062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.432+0000 7f06837fe700 1 -- 172.21.15.100:0/389706575 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f069407f0b0 con 0x7f069c0a9040 2024-08-17T02:22:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.432+0000 7f069affd700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0688060860 0x7f0688062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.432+0000 7f069affd700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0688060860 0x7f0688062d10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f069c0ade60 tx=0x7f06900091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.440+0000 7f06837fe700 1 -- 172.21.15.100:0/389706575 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f069404f500 con 0x7f069c0a9040 2024-08-17T02:22:40.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.604+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f069c06efa0 con 0x7f069c0a9040 2024-08-17T02:22:40.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.604+0000 7f06837fe700 1 -- 172.21.15.100:0/389706575 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0694052b20 con 0x7f069c0a9040 2024-08-17T02:22:40.606 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.604+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0688060860 msgr2=0x7f0688062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.604+0000 7f06a1b99700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0688060860 0x7f0688062d10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f069c0ade60 tx=0x7f06900091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.604+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f069c0a9040 msgr2=0x7f069c152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.604+0000 7f06a1b99700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f069c0a9040 0x7f069c152850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f069400b6c0 tx=0x7f0694019000 comp rx=0 tx=0).stop 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.608+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 shutdown_connections 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.608+0000 7f06a1b99700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0688060860 0x7f0688062d10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.608+0000 7f06a1b99700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c0adf20 0x7f069c0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.608+0000 7f06a1b99700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0acf00 0x7f069c15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.608+0000 7f06a1b99700 1 --2- 172.21.15.100:0/389706575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f069c0a9040 0x7f069c152850 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.608+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 >> 172.21.15.100:0/389706575 conn(0x7f069c066e00 msgr2=0x7f069c06ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:40.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.608+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 shutdown_connections 2024-08-17T02:22:40.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.608+0000 7f06a1b99700 1 -- 172.21.15.100:0/389706575 wait complete. 2024-08-17T02:22:40.620 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8231f27700 1 Processor -- start 2024-08-17T02:22:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8231f27700 1 -- start start 2024-08-17T02:22:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8231f27700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e510 0x7f822c1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8231f27700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f822c13e250 0x7f822c13e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8231f27700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822c156940 con 0x7f822c06e140 2024-08-17T02:22:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8231f27700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822c13ec10 con 0x7f822c13e250 2024-08-17T02:22:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8231f27700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c155d30 con 0x7f822c06e510 2024-08-17T02:22:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8231726700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f822c13e250 0x7f822c13e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f822bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e510 0x7f822c1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8230f25700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f822c06e140 0x7f822c0b1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:58328/0 (socket says 172.21.15.100:58328) 2024-08-17T02:22:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.728+0000 7f8230f25700 1 -- 172.21.15.100:0/1699486683 learned_addr learned my addr 172.21.15.100:0/1699486683 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231726700 1 -- 172.21.15.100:0/1699486683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e510 msgr2=0x7f822c1556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231726700 1 --2- 172.21.15.100:0/1699486683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e510 0x7f822c1556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231726700 1 -- 172.21.15.100:0/1699486683 >> v1:172.21.15.123:6789/0 conn(0x7f822c06e140 legacy=0x7f822c0b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231726700 1 -- 172.21.15.100:0/1699486683 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c156010 con 0x7f822c13e250 2024-08-17T02:22:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231726700 1 --2- 172.21.15.100:0/1699486683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c13e250 0x7f822c13e6d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f821c0098e0 tx=0x7f821c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9534409ede2e6f8d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f822b7fe700 1 -- 172.21.15.100:0/1699486683 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00b640 con 0x7f822c13e250 2024-08-17T02:22:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f822b7fe700 1 -- 172.21.15.100:0/1699486683 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c00b7a0 con 0x7f822c13e250 2024-08-17T02:22:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f822b7fe700 1 -- 172.21.15.100:0/1699486683 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00bb40 con 0x7f822c13e250 2024-08-17T02:22:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231f27700 1 -- 172.21.15.100:0/1699486683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c13e250 msgr2=0x7f822c13e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231f27700 1 --2- 172.21.15.100:0/1699486683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c13e250 0x7f822c13e6d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f821c0098e0 tx=0x7f821c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231f27700 1 -- 172.21.15.100:0/1699486683 shutdown_connections 2024-08-17T02:22:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231f27700 1 --2- 172.21.15.100:0/1699486683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c13e250 0x7f822c13e6d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231f27700 1 --2- 172.21.15.100:0/1699486683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e510 0x7f822c1556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231f27700 1 -- 172.21.15.100:0/1699486683 >> 172.21.15.100:0/1699486683 conn(0x7f822c0678d0 msgr2=0x7f822c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:40.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231f27700 1 -- 172.21.15.100:0/1699486683 shutdown_connections 2024-08-17T02:22:40.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.732+0000 7f8231f27700 1 -- 172.21.15.100:0/1699486683 wait complete. 2024-08-17T02:22:40.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231f27700 1 Processor -- start 2024-08-17T02:22:40.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231f27700 1 -- start start 2024-08-17T02:22:40.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231f27700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e140 0x7f822c130ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231f27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c06e510 0x7f822c137010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231f27700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822c13e250 0x7f822c13c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231f27700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c13a680 con 0x7f822c13e250 2024-08-17T02:22:40.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231f27700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c13bac0 con 0x7f822c06e510 2024-08-17T02:22:40.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231f27700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c13a0c0 con 0x7f822c06e140 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231726700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822c13e250 0x7f822c13c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231726700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822c13e250 0x7f822c13c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50270/0 (socket says 172.21.15.100:50270) 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8231726700 1 -- 172.21.15.100:0/350410612 learned_addr learned my addr 172.21.15.100:0/350410612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f8230f25700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e140 0x7f822c130ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.736+0000 7f822bfff700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c06e510 0x7f822c137010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231726700 1 -- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e140 msgr2=0x7f822c130ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231726700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e140 0x7f822c130ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231726700 1 -- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c06e510 msgr2=0x7f822c137010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231726700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c06e510 0x7f822c137010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231726700 1 -- 172.21.15.100:0/350410612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c009420 con 0x7f822c13e250 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231726700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822c13e250 0x7f822c13c800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f822c06c2d0 tx=0x7f821c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8228ff9700 1 -- 172.21.15.100:0/350410612 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c026450 con 0x7f822c13e250 2024-08-17T02:22:40.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8228ff9700 1 -- 172.21.15.100:0/350410612 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c0265b0 con 0x7f822c13e250 2024-08-17T02:22:40.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c13a3a0 con 0x7f822c13e250 2024-08-17T02:22:40.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8228ff9700 1 -- 172.21.15.100:0/350410612 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c026950 con 0x7f822c13e250 2024-08-17T02:22:40.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c13c300 con 0x7f822c13e250 2024-08-17T02:22:40.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c0643a0 con 0x7f822c13e250 2024-08-17T02:22:40.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.740+0000 7f8228ff9700 1 -- 172.21.15.100:0/350410612 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f821c011420 con 0x7f822c13e250 2024-08-17T02:22:40.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.744+0000 7f8228ff9700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8218060860 0x7f8218062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:40.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.744+0000 7f8228ff9700 1 -- 172.21.15.100:0/350410612 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f821c07e790 con 0x7f822c13e250 2024-08-17T02:22:40.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.744+0000 7f8230f25700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8218060860 0x7f8218062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:40.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.744+0000 7f8230f25700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8218060860 0x7f8218062d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8224002250 tx=0x7f8224009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:40.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.752+0000 7f8228ff9700 1 -- 172.21.15.100:0/350410612 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f821c04ed10 con 0x7f822c13e250 2024-08-17T02:22:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.952+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f822c06f5f0 con 0x7f822c13e250 2024-08-17T02:22:40.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.956+0000 7f8228ff9700 1 -- 172.21.15.100:0/350410612 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f821c052330 con 0x7f822c13e250 2024-08-17T02:22:40.957 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:40.957 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:04.459540+0000","modified":"2024-08-17T02:22:07.136998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:22:40.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8218060860 msgr2=0x7f8218062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8218060860 0x7f8218062d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8224002250 tx=0x7f8224009040 comp rx=0 tx=0).stop 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822c13e250 msgr2=0x7f822c13c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822c13e250 0x7f822c13c800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f822c06c2d0 tx=0x7f821c017000 comp rx=0 tx=0).stop 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 shutdown_connections 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8218060860 0x7f8218062d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822c13e250 0x7f822c13c800 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c06e510 0x7f822c137010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 --2- 172.21.15.100:0/350410612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c06e140 0x7f822c130ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 >> 172.21.15.100:0/350410612 conn(0x7f822c0678d0 msgr2=0x7f822c13d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 shutdown_connections 2024-08-17T02:22:40.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:40.960+0000 7f8231f27700 1 -- 172.21.15.100:0/350410612 wait complete. 2024-08-17T02:22:40.964 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-08-17T02:22:40.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d26c1700 1 Processor -- start 2024-08-17T02:22:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d26c1700 1 -- start start 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d26c1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89cc159330 0x7f89cc15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d26c1700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f89cc0caa30 0x7f89cc0cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d26c1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc15bdd0 con 0x7f89cc15c260 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d26c1700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc15b1c0 con 0x7f89cc0caa30 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d26c1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc15b4a0 con 0x7f89cc159330 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89cb7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f89cc0caa30 0x7f89cc0cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89cbfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89cc159330 0x7f89cc15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d0c5e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f89cc15c260 0x7f89cc15c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:58344/0 (socket says 172.21.15.100:58344) 2024-08-17T02:22:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89d0c5e700 1 -- 172.21.15.100:0/3933627591 learned_addr learned my addr 172.21.15.100:0/3933627591 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89cbfff700 1 -- 172.21.15.100:0/3933627591 >> v2:172.21.15.100:3300/0 conn(0x7f89cc0caa30 msgr2=0x7f89cc0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89cbfff700 1 --2- 172.21.15.100:0/3933627591 >> v2:172.21.15.100:3300/0 conn(0x7f89cc0caa30 0x7f89cc0cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89cbfff700 1 -- 172.21.15.100:0/3933627591 >> v1:172.21.15.123:6789/0 conn(0x7f89cc15c260 legacy=0x7f89cc15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89cbfff700 1 -- 172.21.15.100:0/3933627591 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc15b780 con 0x7f89cc159330 2024-08-17T02:22:41.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89cbfff700 1 --2- 172.21.15.100:0/3933627591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 0x7f89cc15aac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f89cc067d60 tx=0x7f89c4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7338a76d4c686b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89caffd700 1 -- 172.21.15.100:0/3933627591 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400e070 con 0x7f89cc159330 2024-08-17T02:22:41.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89caffd700 1 -- 172.21.15.100:0/3933627591 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c400bb60 con 0x7f89cc159330 2024-08-17T02:22:41.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.080+0000 7f89caffd700 1 -- 172.21.15.100:0/3933627591 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4012430 con 0x7f89cc159330 2024-08-17T02:22:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 -- 172.21.15.100:0/3933627591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 msgr2=0x7f89cc15aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 --2- 172.21.15.100:0/3933627591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 0x7f89cc15aac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f89cc067d60 tx=0x7f89c4009780 comp rx=0 tx=0).stop 2024-08-17T02:22:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 -- 172.21.15.100:0/3933627591 shutdown_connections 2024-08-17T02:22:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 --2- 172.21.15.100:0/3933627591 >> v2:172.21.15.100:3300/0 conn(0x7f89cc0caa30 0x7f89cc0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 --2- 172.21.15.100:0/3933627591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 0x7f89cc15aac0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 -- 172.21.15.100:0/3933627591 >> 172.21.15.100:0/3933627591 conn(0x7f89cc0994c0 msgr2=0x7f89cc067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 -- 172.21.15.100:0/3933627591 shutdown_connections 2024-08-17T02:22:41.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 -- 172.21.15.100:0/3933627591 wait complete. 2024-08-17T02:22:41.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 Processor -- start 2024-08-17T02:22:41.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.084+0000 7f89d26c1700 1 -- start start 2024-08-17T02:22:41.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d26c1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89cc0caa30 0x7f89cc155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d26c1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 0x7f89cc0b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d26c1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89cc15c260 0x7f89cc0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d26c1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc156390 con 0x7f89cc15c260 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d26c1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0c9f50 con 0x7f89cc159330 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d26c1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0c93a0 con 0x7f89cc0caa30 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89cc15c260 0x7f89cc0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89cbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89cc0caa30 0x7f89cc155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89cc15c260 0x7f89cc0b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50300/0 (socket says 172.21.15.100:50300) 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 -- 172.21.15.100:0/2089397813 learned_addr learned my addr 172.21.15.100:0/2089397813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89cb7fe700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 0x7f89cc0b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 -- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89cc0caa30 msgr2=0x7f89cc155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89cc0caa30 0x7f89cc155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 -- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 msgr2=0x7f89cc0b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 0x7f89cc0b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 -- 172.21.15.100:0/2089397813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c4009a20 con 0x7f89cc15c260 2024-08-17T02:22:41.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d0c5e700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89cc15c260 0x7f89cc0b44b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00c4a0 tx=0x7f89bc007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89b3fff700 1 -- 172.21.15.100:0/2089397813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00ce50 con 0x7f89cc15c260 2024-08-17T02:22:41.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89b3fff700 1 -- 172.21.15.100:0/2089397813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc005b30 con 0x7f89cc15c260 2024-08-17T02:22:41.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc0c9620 con 0x7f89cc15c260 2024-08-17T02:22:41.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89b3fff700 1 -- 172.21.15.100:0/2089397813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0070d0 con 0x7f89cc15c260 2024-08-17T02:22:41.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.088+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc0c9be0 con 0x7f89cc15c260 2024-08-17T02:22:41.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.092+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc0ca0e0 con 0x7f89cc15c260 2024-08-17T02:22:41.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.092+0000 7f89b3fff700 1 -- 172.21.15.100:0/2089397813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f89bc0074c0 con 0x7f89cc15c260 2024-08-17T02:22:41.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.092+0000 7f89b3fff700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89b8060860 0x7f89b8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.092+0000 7f89b3fff700 1 -- 172.21.15.100:0/2089397813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89bc07be20 con 0x7f89cc15c260 2024-08-17T02:22:41.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.092+0000 7f89cbfff700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89b8060860 0x7f89b8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.092+0000 7f89cbfff700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89b8060860 0x7f89b8062d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f89c40097c0 tx=0x7f89c4019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.100+0000 7f89b3fff700 1 -- 172.21.15.100:0/2089397813 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f89bc04c270 con 0x7f89cc15c260 2024-08-17T02:22:41.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.268+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89cc0645c0 con 0x7f89cc15c260 2024-08-17T02:22:41.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.268+0000 7f89b3fff700 1 -- 172.21.15.100:0/2089397813 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f89bc04f890 con 0x7f89cc15c260 2024-08-17T02:22:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:41.272 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89b8060860 msgr2=0x7f89b8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89b8060860 0x7f89b8062d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f89c40097c0 tx=0x7f89c4019000 comp rx=0 tx=0).stop 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89cc15c260 msgr2=0x7f89cc0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89cc15c260 0x7f89cc0b44b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00c4a0 tx=0x7f89bc007f10 comp rx=0 tx=0).stop 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 shutdown_connections 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f89b8060860 0x7f89b8062d10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89cc15c260 0x7f89cc0b44b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89cc159330 0x7f89cc0b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 --2- 172.21.15.100:0/2089397813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89cc0caa30 0x7f89cc155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 >> 172.21.15.100:0/2089397813 conn(0x7f89cc0994c0 msgr2=0x7f89cc157280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:41.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 shutdown_connections 2024-08-17T02:22:41.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.272+0000 7f89d26c1700 1 -- 172.21.15.100:0/2089397813 wait complete. 2024-08-17T02:22:41.287 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-17T02:22:41.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:41.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 Processor -- start 2024-08-17T02:22:41.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- start start 2024-08-17T02:22:41.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb15815d540 0x7fb1580af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb15806e830 0x7fb1580a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb1580a9260 0x7fb1580b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1580b0270 con 0x7fb15806e830 2024-08-17T02:22:41.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1580af660 con 0x7fb1580a9260 2024-08-17T02:22:41.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1580af940 con 0x7fb15815d540 2024-08-17T02:22:41.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb15806e830 0x7fb1580a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb15806e830 0x7fb1580a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50316/0 (socket says 172.21.15.100:50316) 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 -- 172.21.15.100:0/394360535 learned_addr learned my addr 172.21.15.100:0/394360535 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 -- 172.21.15.100:0/394360535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb15815d540 msgr2=0x7fb1580af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15cd01700 1 --2- 172.21.15.100:0/394360535 >> v2:172.21.15.123:3300/0 conn(0x7fb1580a9260 0x7fb1580b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb157fff700 1 --2- 172.21.15.100:0/394360535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb15815d540 0x7fb1580af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 --2- 172.21.15.100:0/394360535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb15815d540 0x7fb1580af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 -- 172.21.15.100:0/394360535 >> v2:172.21.15.123:3300/0 conn(0x7fb1580a9260 msgr2=0x7fb1580b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 --2- 172.21.15.100:0/394360535 >> v2:172.21.15.123:3300/0 conn(0x7fb1580a9260 0x7fb1580b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 -- 172.21.15.100:0/394360535 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580afc20 con 0x7fb15806e830 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15cd01700 1 --2- 172.21.15.100:0/394360535 >> v2:172.21.15.123:3300/0 conn(0x7fb1580a9260 0x7fb1580b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb157fff700 1 --2- 172.21.15.100:0/394360535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb15815d540 0x7fb1580af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb1577fe700 1 --2- 172.21.15.100:0/394360535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15806e830 0x7fb1580a8d20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb1581623b0 tx=0x7fb148009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a1d29dd7903537 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb156ffd700 1 -- 172.21.15.100:0/394360535 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800e070 con 0x7fb15806e830 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb156ffd700 1 -- 172.21.15.100:0/394360535 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800ba50 con 0x7fb15806e830 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb156ffd700 1 -- 172.21.15.100:0/394360535 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1480092e0 con 0x7fb15806e830 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- 172.21.15.100:0/394360535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15806e830 msgr2=0x7fb1580a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 --2- 172.21.15.100:0/394360535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15806e830 0x7fb1580a8d20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb1581623b0 tx=0x7fb148009840 comp rx=0 tx=0).stop 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- 172.21.15.100:0/394360535 shutdown_connections 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 --2- 172.21.15.100:0/394360535 >> v2:172.21.15.123:3300/0 conn(0x7fb1580a9260 0x7fb1580b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 --2- 172.21.15.100:0/394360535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15806e830 0x7fb1580a8d20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 --2- 172.21.15.100:0/394360535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb15815d540 0x7fb1580af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- 172.21.15.100:0/394360535 >> 172.21.15.100:0/394360535 conn(0x7fb158066e00 msgr2=0x7fb158067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- 172.21.15.100:0/394360535 shutdown_connections 2024-08-17T02:22:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.392+0000 7fb15e764700 1 -- 172.21.15.100:0/394360535 wait complete. 2024-08-17T02:22:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 Processor -- start 2024-08-17T02:22:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 -- start start 2024-08-17T02:22:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb15806e830 0x7fb1581580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1580a9260 0x7fb158099280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15815d540 0x7fb15815bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb158098fb0 con 0x7fb15815d540 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb158158990 con 0x7fb15806e830 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15816e300 con 0x7fb1580a9260 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15815d540 0x7fb15815bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15815d540 0x7fb15815bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50324/0 (socket says 172.21.15.100:50324) 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 -- 172.21.15.100:0/1140756560 learned_addr learned my addr 172.21.15.100:0/1140756560 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb157fff700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb15806e830 0x7fb1581580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb1577fe700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1580a9260 0x7fb158099280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 -- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1580a9260 msgr2=0x7fb158099280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1580a9260 0x7fb158099280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 -- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb15806e830 msgr2=0x7fb1581580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb15806e830 0x7fb1581580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 -- 172.21.15.100:0/1140756560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb148009ae0 con 0x7fb15815d540 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb157fff700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb15806e830 0x7fb1581580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15cd01700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15815d540 0x7fb15815bf00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb15000c8b0 tx=0x7fb150008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb13ffff700 1 -- 172.21.15.100:0/1140756560 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150011600 con 0x7fb15815d540 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb13ffff700 1 -- 172.21.15.100:0/1140756560 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb150011780 con 0x7fb15815d540 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb158098cd0 con 0x7fb15815d540 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb13ffff700 1 -- 172.21.15.100:0/1140756560 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150011b40 con 0x7fb15815d540 2024-08-17T02:22:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.396+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15816df90 con 0x7fb15815d540 2024-08-17T02:22:41.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.400+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1580990f0 con 0x7fb15815d540 2024-08-17T02:22:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.400+0000 7fb13ffff700 1 -- 172.21.15.100:0/1140756560 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb1500223a0 con 0x7fb15815d540 2024-08-17T02:22:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.400+0000 7fb13ffff700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb144060860 0x7fb144062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.400+0000 7fb13ffff700 1 -- 172.21.15.100:0/1140756560 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb15007ce90 con 0x7fb15815d540 2024-08-17T02:22:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.400+0000 7fb157fff700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb144060860 0x7fb144062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.400+0000 7fb157fff700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb144060860 0x7fb144062d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb14c006fd0 tx=0x7fb14c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.408+0000 7fb13ffff700 1 -- 172.21.15.100:0/1140756560 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb150050d70 con 0x7fb15815d540 2024-08-17T02:22:41.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.600+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1580645c0 con 0x7fb15815d540 2024-08-17T02:22:41.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb13ffff700 1 -- 172.21.15.100:0/1140756560 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fb150050900 con 0x7fb15815d540 2024-08-17T02:22:41.606 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:41.606 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:04.459540+0000","modified":"2024-08-17T02:22:07.136998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:22:41.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb144060860 msgr2=0x7fb144062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb144060860 0x7fb144062d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb14c006fd0 tx=0x7fb14c008040 comp rx=0 tx=0).stop 2024-08-17T02:22:41.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15815d540 msgr2=0x7fb15815bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15815d540 0x7fb15815bf00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb15000c8b0 tx=0x7fb150008040 comp rx=0 tx=0).stop 2024-08-17T02:22:41.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 shutdown_connections 2024-08-17T02:22:41.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb144060860 0x7fb144062d10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb15815d540 0x7fb15815bf00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1580a9260 0x7fb158099280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 --2- 172.21.15.100:0/1140756560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb15806e830 0x7fb1581580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 >> 172.21.15.100:0/1140756560 conn(0x7fb158066e00 msgr2=0x7fb15806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:41.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 shutdown_connections 2024-08-17T02:22:41.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.604+0000 7fb15e764700 1 -- 172.21.15.100:0/1140756560 wait complete. 2024-08-17T02:22:41.611 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-08-17T02:22:41.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:41.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.728+0000 7f2e06c93700 1 Processor -- start 2024-08-17T02:22:41.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.728+0000 7f2e06c93700 1 -- start start 2024-08-17T02:22:41.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000af5a0 0x7f2e000af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2e000aff10 0x7f2e00155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e000a9f60 con 0x7f2e00155380 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0006e4b0 con 0x7f2e000aff10 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0006e6a0 con 0x7f2e000af5a0 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e04a2f700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2e00155380 0x7f2e000a9730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57898/0 (socket says 172.21.15.100:57898) 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e04a2f700 1 -- 172.21.15.100:0/446710670 learned_addr learned my addr 172.21.15.100:0/446710670 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e05230700 1 --2- 172.21.15.100:0/446710670 >> v2:172.21.15.100:3300/0 conn(0x7f2e000aff10 0x7f2e00155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2dfffff700 1 --2- 172.21.15.100:0/446710670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000af5a0 0x7f2e000af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2dff7fe700 1 -- 172.21.15.100:0/446710670 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627028768 0 0) 0x7f2e000a9f60 con 0x7f2e00155380 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2dff7fe700 1 -- 172.21.15.100:0/446710670 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de8003570 con 0x7f2e00155380 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e05230700 1 -- 172.21.15.100:0/446710670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000af5a0 msgr2=0x7f2e000af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e05230700 1 --2- 172.21.15.100:0/446710670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000af5a0 0x7f2e000af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e05230700 1 -- 172.21.15.100:0/446710670 >> v1:172.21.15.100:6789/0 conn(0x7f2e00155380 legacy=0x7f2e000a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e05230700 1 -- 172.21.15.100:0/446710670 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0006e980 con 0x7f2e000aff10 2024-08-17T02:22:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2dfffff700 1 --2- 172.21.15.100:0/446710670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000af5a0 0x7f2e000af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e05230700 1 --2- 172.21.15.100:0/446710670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e000aff10 0x7f2e00155750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e60 tx=0x7f2df8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1c6e1d60919fa0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2dff7fe700 1 -- 172.21.15.100:0/446710670 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800e070 con 0x7f2e000aff10 2024-08-17T02:22:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2dff7fe700 1 -- 172.21.15.100:0/446710670 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df800bab0 con 0x7f2e000aff10 2024-08-17T02:22:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2dff7fe700 1 -- 172.21.15.100:0/446710670 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800be30 con 0x7f2e000aff10 2024-08-17T02:22:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 -- 172.21.15.100:0/446710670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e000aff10 msgr2=0x7f2e00155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 --2- 172.21.15.100:0/446710670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e000aff10 0x7f2e00155750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e60 tx=0x7f2df8009780 comp rx=0 tx=0).stop 2024-08-17T02:22:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 -- 172.21.15.100:0/446710670 shutdown_connections 2024-08-17T02:22:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 --2- 172.21.15.100:0/446710670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e000aff10 0x7f2e00155750 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 --2- 172.21.15.100:0/446710670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000af5a0 0x7f2e000af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 -- 172.21.15.100:0/446710670 >> 172.21.15.100:0/446710670 conn(0x7f2e00066e00 msgr2=0x7f2e00067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 -- 172.21.15.100:0/446710670 shutdown_connections 2024-08-17T02:22:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.732+0000 7f2e06c93700 1 -- 172.21.15.100:0/446710670 wait complete. 2024-08-17T02:22:41.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.736+0000 7f2e06c93700 1 Processor -- start 2024-08-17T02:22:41.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.736+0000 7f2e06c93700 1 -- start start 2024-08-17T02:22:41.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e000af5a0 0x7f2e000c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000aff10 0x7f2e000c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e00155380 0x7f2e00164e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000c21b0 con 0x7f2e00155380 2024-08-17T02:22:41.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000c15a0 con 0x7f2e000af5a0 2024-08-17T02:22:41.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000c1880 con 0x7f2e000aff10 2024-08-17T02:22:41.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e00155380 0x7f2e00164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e00155380 0x7f2e00164e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50342/0 (socket says 172.21.15.100:50342) 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 -- 172.21.15.100:0/476952939 learned_addr learned my addr 172.21.15.100:0/476952939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2dfffff700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000aff10 0x7f2e000c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e04a2f700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e000af5a0 0x7f2e000c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 -- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000aff10 msgr2=0x7f2e000c0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000aff10 0x7f2e000c0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 -- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e000af5a0 msgr2=0x7f2e000c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e000af5a0 0x7f2e000c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 -- 172.21.15.100:0/476952939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df400a5d0 con 0x7f2e00155380 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e04a2f700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e000af5a0 0x7f2e000c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e05230700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e00155380 0x7f2e00164e40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e0006f870 tx=0x7f2df8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2dfcff9700 1 -- 172.21.15.100:0/476952939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8027470 con 0x7f2e00155380 2024-08-17T02:22:41.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2dfcff9700 1 -- 172.21.15.100:0/476952939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8027610 con 0x7f2e00155380 2024-08-17T02:22:41.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df8009a20 con 0x7f2e00155380 2024-08-17T02:22:41.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2dfcff9700 1 -- 172.21.15.100:0/476952939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80279d0 con 0x7f2e00155380 2024-08-17T02:22:41.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e000c1e40 con 0x7f2e00155380 2024-08-17T02:22:41.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e000c2360 con 0x7f2e00155380 2024-08-17T02:22:41.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2dfcff9700 1 -- 172.21.15.100:0/476952939 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2df8024070 con 0x7f2e00155380 2024-08-17T02:22:41.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2dfcff9700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2dec060860 0x7f2dec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:41.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.740+0000 7f2dfcff9700 1 -- 172.21.15.100:0/476952939 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2df807f500 con 0x7f2e00155380 2024-08-17T02:22:41.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.744+0000 7f2e04a2f700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2dec060860 0x7f2dec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:41.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.744+0000 7f2e04a2f700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2dec060860 0x7f2dec062d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2df00058d0 tx=0x7f2df0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:41.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.752+0000 7f2dfcff9700 1 -- 172.21.15.100:0/476952939 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2e000c2360 con 0x7f2e00155380 2024-08-17T02:22:41.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.944+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e000645c0 con 0x7f2e00155380 2024-08-17T02:22:41.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.944+0000 7f2dfcff9700 1 -- 172.21.15.100:0/476952939 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f2df804f950 con 0x7f2e00155380 2024-08-17T02:22:41.948 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:41.948 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3896113220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3896113220},{"type":"v1","addr":"172.21.15.123:6835","nonce":3896113220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2780049234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2780049234},{"type":"v1","addr":"172.21.15.100:6833","nonce":2780049234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4121836367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4121836367},{"type":"v1","addr":"172.21.15.100:6835","nonce":4121836367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:04.459540+0000","modified":"2024-08-17T02:22:07.136998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2524989904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2524989904},{"type":"v1","addr":"172.21.15.123:6837","nonce":2524989904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:22:41.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2dec060860 msgr2=0x7f2dec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2dec060860 0x7f2dec062d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2df00058d0 tx=0x7f2df0009040 comp rx=0 tx=0).stop 2024-08-17T02:22:41.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e00155380 msgr2=0x7f2e00164e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:41.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e00155380 0x7f2e00164e40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e0006f870 tx=0x7f2df8019000 comp rx=0 tx=0).stop 2024-08-17T02:22:41.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 shutdown_connections 2024-08-17T02:22:41.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2dec060860 0x7f2dec062d10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e00155380 0x7f2e00164e40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e000aff10 0x7f2e000c0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 --2- 172.21.15.100:0/476952939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e000af5a0 0x7f2e000c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:41.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 >> 172.21.15.100:0/476952939 conn(0x7f2e00066e00 msgr2=0x7f2e000ab850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:41.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 shutdown_connections 2024-08-17T02:22:41.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:41.948+0000 7f2e06c93700 1 -- 172.21.15.100:0/476952939 wait complete. 2024-08-17T02:22:41.951 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-08-17T02:22:41.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:42.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f25a57cd700 1 Processor -- start 2024-08-17T02:22:42.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f25a57cd700 1 -- start start 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f25a57cd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25a0142860 0x7f25a006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f25a57cd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25a0142c30 0x7f25a006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f25a57cd700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a00b2220 con 0x7f25a006f2d0 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f25a57cd700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a0159790 con 0x7f25a0142c30 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f25a57cd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00b2500 con 0x7f25a0142860 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f259f7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25a0142c30 0x7f25a006ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f259f7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25a0142c30 0x7f25a006ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50350/0 (socket says 172.21.15.100:50350) 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f259f7fe700 1 -- 172.21.15.100:0/4272456565 learned_addr learned my addr 172.21.15.100:0/4272456565 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.068+0000 7f25a4fcc700 1 --1- 172.21.15.100:0/4272456565 >> v1:172.21.15.100:6789/0 conn(0x7f25a006f2d0 0x7f25a00b0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57900/0 (socket says 172.21.15.100:57900) 2024-08-17T02:22:42.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259ffff700 1 --2- 172.21.15.100:0/4272456565 >> v2:172.21.15.123:3300/0 conn(0x7f25a0142860 0x7f25a006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259f7fe700 1 -- 172.21.15.100:0/4272456565 >> v2:172.21.15.123:3300/0 conn(0x7f25a0142860 msgr2=0x7f25a006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:42.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259f7fe700 1 --2- 172.21.15.100:0/4272456565 >> v2:172.21.15.123:3300/0 conn(0x7f25a0142860 0x7f25a006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259f7fe700 1 -- 172.21.15.100:0/4272456565 >> v1:172.21.15.100:6789/0 conn(0x7f25a006f2d0 legacy=0x7f25a00b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259effd700 1 -- 172.21.15.100:0/4272456565 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737155242 0 0) 0x7f25a00b2220 con 0x7f25a006f2d0 2024-08-17T02:22:42.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259f7fe700 1 -- 172.21.15.100:0/4272456565 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00b27e0 con 0x7f25a0142c30 2024-08-17T02:22:42.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259f7fe700 1 --2- 172.21.15.100:0/4272456565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a0142c30 0x7f25a006ec60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25a006d110 tx=0x7f2588009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbaaa049cf61498 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259effd700 1 -- 172.21.15.100:0/4272456565 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258800e070 con 0x7f25a0142c30 2024-08-17T02:22:42.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259effd700 1 -- 172.21.15.100:0/4272456565 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258800ba50 con 0x7f25a0142c30 2024-08-17T02:22:42.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f259effd700 1 -- 172.21.15.100:0/4272456565 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880092e0 con 0x7f25a0142c30 2024-08-17T02:22:42.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f25a57cd700 1 -- 172.21.15.100:0/4272456565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a0142c30 msgr2=0x7f25a006ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f25a57cd700 1 --2- 172.21.15.100:0/4272456565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a0142c30 0x7f25a006ec60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25a006d110 tx=0x7f2588009840 comp rx=0 tx=0).stop 2024-08-17T02:22:42.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f25a57cd700 1 -- 172.21.15.100:0/4272456565 shutdown_connections 2024-08-17T02:22:42.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f25a57cd700 1 --2- 172.21.15.100:0/4272456565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a0142c30 0x7f25a006ec60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f25a57cd700 1 --2- 172.21.15.100:0/4272456565 >> v2:172.21.15.123:3300/0 conn(0x7f25a0142860 0x7f25a006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f25a57cd700 1 -- 172.21.15.100:0/4272456565 >> 172.21.15.100:0/4272456565 conn(0x7f25a00ca2b0 msgr2=0x7f25a006e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:42.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f25a57cd700 1 -- 172.21.15.100:0/4272456565 shutdown_connections 2024-08-17T02:22:42.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.072+0000 7f25a57cd700 1 -- 172.21.15.100:0/4272456565 wait complete. 2024-08-17T02:22:42.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 Processor -- start 2024-08-17T02:22:42.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 -- start start 2024-08-17T02:22:42.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a006f2d0 0x7f25a00c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25a0142860 0x7f25a0158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25a0142c30 0x7f25a0158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a015d990 con 0x7f25a006f2d0 2024-08-17T02:22:42.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a015dc70 con 0x7f25a0142c30 2024-08-17T02:22:42.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a015df50 con 0x7f25a0142860 2024-08-17T02:22:42.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a006f2d0 0x7f25a00c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a006f2d0 0x7f25a00c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50356/0 (socket says 172.21.15.100:50356) 2024-08-17T02:22:42.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 -- 172.21.15.100:0/2826662612 learned_addr learned my addr 172.21.15.100:0/2826662612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:42.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a4fcc700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25a0142c30 0x7f25a0158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259f7fe700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25a0142860 0x7f25a0158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 -- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25a0142860 msgr2=0x7f25a0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25a0142860 0x7f25a0158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 -- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25a0142c30 msgr2=0x7f25a0158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25a0142c30 0x7f25a0158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 -- 172.21.15.100:0/2826662612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588009ae0 con 0x7f25a006f2d0 2024-08-17T02:22:42.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f259ffff700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a006f2d0 0x7f25a00c5660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f259800d890 tx=0x7f2598008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f2597fff700 1 -- 172.21.15.100:0/2826662612 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980125f0 con 0x7f25a006f2d0 2024-08-17T02:22:42.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f2597fff700 1 -- 172.21.15.100:0/2826662612 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2598012750 con 0x7f25a006f2d0 2024-08-17T02:22:42.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a015e230 con 0x7f25a006f2d0 2024-08-17T02:22:42.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f2597fff700 1 -- 172.21.15.100:0/2826662612 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598012af0 con 0x7f25a006f2d0 2024-08-17T02:22:42.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.076+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a013f0b0 con 0x7f25a006f2d0 2024-08-17T02:22:42.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.080+0000 7f2597fff700 1 -- 172.21.15.100:0/2826662612 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f259801e460 con 0x7f25a006f2d0 2024-08-17T02:22:42.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.080+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a00643a0 con 0x7f25a006f2d0 2024-08-17T02:22:42.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.080+0000 7f2597fff700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2584060860 0x7f2584062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.080+0000 7f2597fff700 1 -- 172.21.15.100:0/2826662612 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f259807d120 con 0x7f25a006f2d0 2024-08-17T02:22:42.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.080+0000 7f259f7fe700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2584060860 0x7f2584062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.080+0000 7f259f7fe700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2584060860 0x7f2584062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2588009880 tx=0x7f2588018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.088+0000 7f2597fff700 1 -- 172.21.15.100:0/2826662612 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2598051000 con 0x7f25a006f2d0 2024-08-17T02:22:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.252+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25a00708e0 con 0x7f25a006f2d0 2024-08-17T02:22:42.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f2597fff700 1 -- 172.21.15.100:0/2826662612 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2598050b90 con 0x7f25a006f2d0 2024-08-17T02:22:42.257 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:42.257 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:06.126600+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:02.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:03.458625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2584060860 msgr2=0x7f2584062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2584060860 0x7f2584062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2588009880 tx=0x7f2588018000 comp rx=0 tx=0).stop 2024-08-17T02:22:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a006f2d0 msgr2=0x7f25a00c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a006f2d0 0x7f25a00c5660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f259800d890 tx=0x7f2598008040 comp rx=0 tx=0).stop 2024-08-17T02:22:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 shutdown_connections 2024-08-17T02:22:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2584060860 0x7f2584062d10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25a0142c30 0x7f25a0158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25a0142860 0x7f25a0158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 --2- 172.21.15.100:0/2826662612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25a006f2d0 0x7f25a00c5660 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 >> 172.21.15.100:0/2826662612 conn(0x7f25a00ca2b0 msgr2=0x7f25a00b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 shutdown_connections 2024-08-17T02:22:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.256+0000 7f25a57cd700 1 -- 172.21.15.100:0/2826662612 wait complete. 2024-08-17T02:22:42.272 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-17T02:22:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e4ad85700 1 Processor -- start 2024-08-17T02:22:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e4ad85700 1 -- start start 2024-08-17T02:22:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e4ad85700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e44159340 0x7f5e44159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e4ad85700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 0x7f5e440b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e4ad85700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e440b1cf0 con 0x7f5e44159c50 2024-08-17T02:22:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e4ad85700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e440b60d0 con 0x7f5e44159340 2024-08-17T02:22:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e4ad85700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b10e0 con 0x7f5e440b4330 2024-08-17T02:22:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e48b21700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e44159340 0x7f5e44159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e48b21700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e44159340 0x7f5e44159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50364/0 (socket says 172.21.15.100:50364) 2024-08-17T02:22:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e48b21700 1 -- 172.21.15.100:0/3030364126 learned_addr learned my addr 172.21.15.100:0/3030364126 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.376+0000 7f5e49322700 1 --2- 172.21.15.100:0/3030364126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 0x7f5e440b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e48b21700 1 -- 172.21.15.100:0/3030364126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 msgr2=0x7f5e440b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e48b21700 1 --2- 172.21.15.100:0/3030364126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 0x7f5e440b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e48b21700 1 -- 172.21.15.100:0/3030364126 >> v1:172.21.15.123:6789/0 conn(0x7f5e44159c50 legacy=0x7f5e440b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e48b21700 1 -- 172.21.15.100:0/3030364126 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440b13c0 con 0x7f5e44159340 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e48b21700 1 --2- 172.21.15.100:0/3030364126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e44159340 0x7f5e44159710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5e340099b0 tx=0x7f5e34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7e76615f23801b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e437fe700 1 -- 172.21.15.100:0/3030364126 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34009ea0 con 0x7f5e44159340 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e437fe700 1 -- 172.21.15.100:0/3030364126 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3400b860 con 0x7f5e44159340 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e437fe700 1 -- 172.21.15.100:0/3030364126 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400bc00 con 0x7f5e44159340 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3030364126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e44159340 msgr2=0x7f5e44159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3030364126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e44159340 0x7f5e44159710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5e340099b0 tx=0x7f5e34009250 comp rx=0 tx=0).stop 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3030364126 shutdown_connections 2024-08-17T02:22:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3030364126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 0x7f5e440b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3030364126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e44159340 0x7f5e44159710 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5e340099b0 tx=0x7f5e34009250 comp rx=0 tx=0).stop 2024-08-17T02:22:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3030364126 >> 172.21.15.100:0/3030364126 conn(0x7f5e440994f0 msgr2=0x7f5e440675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3030364126 shutdown_connections 2024-08-17T02:22:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3030364126 wait complete. 2024-08-17T02:22:42.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 Processor -- start 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- start start 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 0x7f5e4409e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e44159c50 0x7f5e44156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e440a9d40 0x7f5e4416ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4409ec20 con 0x7f5e440a9d40 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440aedb0 con 0x7f5e44159c50 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e4ad85700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440ae480 con 0x7f5e440b4330 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e49322700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e440a9d40 0x7f5e4416ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e49322700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e440a9d40 0x7f5e4416ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50376/0 (socket says 172.21.15.100:50376) 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.380+0000 7f5e49322700 1 -- 172.21.15.100:0/3661062585 learned_addr learned my addr 172.21.15.100:0/3661062585 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e48b21700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 0x7f5e4409e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e43fff700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e44159c50 0x7f5e44156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e49322700 1 -- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 msgr2=0x7f5e4409e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e49322700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 0x7f5e4409e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e49322700 1 -- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e44159c50 msgr2=0x7f5e44156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e49322700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e44159c50 0x7f5e44156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e49322700 1 -- 172.21.15.100:0/3661062585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e340094f0 con 0x7f5e440a9d40 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e43fff700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e44159c50 0x7f5e44156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e49322700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e440a9d40 0x7f5e4416ca80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009ef0 tx=0x7f5e3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e40ff9700 1 -- 172.21.15.100:0/3661062585 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00cc50 con 0x7f5e440a9d40 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e40ff9700 1 -- 172.21.15.100:0/3661062585 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c00f040 con 0x7f5e440a9d40 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e440ae760 con 0x7f5e440a9d40 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e40ff9700 1 -- 172.21.15.100:0/3661062585 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c014480 con 0x7f5e440a9d40 2024-08-17T02:22:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4409e930 con 0x7f5e440a9d40 2024-08-17T02:22:42.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e440645c0 con 0x7f5e440a9d40 2024-08-17T02:22:42.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e40ff9700 1 -- 172.21.15.100:0/3661062585 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5e3c014830 con 0x7f5e440a9d40 2024-08-17T02:22:42.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e40ff9700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e30060860 0x7f5e30062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.384+0000 7f5e40ff9700 1 -- 172.21.15.100:0/3661062585 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e3c07d6d0 con 0x7f5e440a9d40 2024-08-17T02:22:42.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.388+0000 7f5e48b21700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e30060860 0x7f5e30062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.388+0000 7f5e48b21700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e30060860 0x7f5e30062d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5e340099b0 tx=0x7f5e34017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.388+0000 7f5e40ff9700 1 -- 172.21.15.100:0/3661062585 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5e3c04dbd0 con 0x7f5e440a9d40 2024-08-17T02:22:42.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.588+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5e440b4760 con 0x7f5e440a9d40 2024-08-17T02:22:42.593 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-08-17T02:22:42.592+0000 7f6fd553c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-17T02:22:42.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e40ff9700 1 -- 172.21.15.100:0/3661062585 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5e3c018090 con 0x7f5e440a9d40 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e30060860 msgr2=0x7f5e30062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e30060860 0x7f5e30062d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5e340099b0 tx=0x7f5e34017000 comp rx=0 tx=0).stop 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e440a9d40 msgr2=0x7f5e4416ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e440a9d40 0x7f5e4416ca80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009ef0 tx=0x7f5e3c00b040 comp rx=0 tx=0).stop 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 shutdown_connections 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e30060860 0x7f5e30062d10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e440a9d40 0x7f5e4416ca80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e44159c50 0x7f5e44156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 --2- 172.21.15.100:0/3661062585 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440b4330 0x7f5e4409e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 >> 172.21.15.100:0/3661062585 conn(0x7f5e440994f0 msgr2=0x7f5e4415a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:42.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 shutdown_connections 2024-08-17T02:22:42.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.600+0000 7f5e4ad85700 1 -- 172.21.15.100:0/3661062585 wait complete. 2024-08-17T02:22:42.606 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-17T02:22:42.607 INFO:tasks.ceph.mds.d.smithi123.stderr: -2264> 2024-08-17T02:22:09.446+0000 7f8f96042700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:22:42.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-17T02:22:42.634 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2024-08-17T02:22:42.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6353fff700 1 Processor -- start 2024-08-17T02:22:42.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6353fff700 1 -- start start 2024-08-17T02:22:42.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6353fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6354067030 0x7f6354067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6353fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63540cfb00 0x7f63540cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6353fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f63540cb5c0 0x7f6354158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6353fff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63540cf670 con 0x7f63540cb5c0 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6353fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63540cea60 con 0x7f6354067030 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6353fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63540ced40 con 0x7f63540cfb00 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f63537fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f63540cb5c0 0x7f6354158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f63537fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f63540cb5c0 0x7f6354158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50388/0 (socket says 172.21.15.100:50388) 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f63537fe700 1 -- 172.21.15.100:0/2488635019 learned_addr learned my addr 172.21.15.100:0/2488635019 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f63527fc700 1 --2- 172.21.15.100:0/2488635019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63540cfb00 0x7f63540cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f6352ffd700 1 --2- 172.21.15.100:0/2488635019 >> v2:172.21.15.123:3300/0 conn(0x7f6354067030 0x7f6354067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f63537fe700 1 -- 172.21.15.100:0/2488635019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63540cfb00 msgr2=0x7f63540cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:42.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.720+0000 7f63537fe700 1 --2- 172.21.15.100:0/2488635019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63540cfb00 0x7f63540cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f63537fe700 1 -- 172.21.15.100:0/2488635019 >> v2:172.21.15.123:3300/0 conn(0x7f6354067030 msgr2=0x7f6354067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:42.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f63537fe700 1 --2- 172.21.15.100:0/2488635019 >> v2:172.21.15.123:3300/0 conn(0x7f6354067030 0x7f6354067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f63537fe700 1 -- 172.21.15.100:0/2488635019 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540cf020 con 0x7f63540cb5c0 2024-08-17T02:22:42.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f63537fe700 1 --2- 172.21.15.100:0/2488635019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cb5c0 0x7f6354158cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63540d1070 tx=0x7f634c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a20f9ad6d4e5e4b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6351ffb700 1 -- 172.21.15.100:0/2488635019 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c00e070 con 0x7f63540cb5c0 2024-08-17T02:22:42.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6351ffb700 1 -- 172.21.15.100:0/2488635019 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c00b970 con 0x7f63540cb5c0 2024-08-17T02:22:42.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6351ffb700 1 -- 172.21.15.100:0/2488635019 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c0092e0 con 0x7f63540cb5c0 2024-08-17T02:22:42.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 -- 172.21.15.100:0/2488635019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cb5c0 msgr2=0x7f6354158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 --2- 172.21.15.100:0/2488635019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cb5c0 0x7f6354158cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63540d1070 tx=0x7f634c009840 comp rx=0 tx=0).stop 2024-08-17T02:22:42.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 -- 172.21.15.100:0/2488635019 shutdown_connections 2024-08-17T02:22:42.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 --2- 172.21.15.100:0/2488635019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cb5c0 0x7f6354158cc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 --2- 172.21.15.100:0/2488635019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63540cfb00 0x7f63540cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 --2- 172.21.15.100:0/2488635019 >> v2:172.21.15.123:3300/0 conn(0x7f6354067030 0x7f6354067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 -- 172.21.15.100:0/2488635019 >> 172.21.15.100:0/2488635019 conn(0x7f63540d0b50 msgr2=0x7f63540ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:42.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 -- 172.21.15.100:0/2488635019 shutdown_connections 2024-08-17T02:22:42.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 -- 172.21.15.100:0/2488635019 wait complete. 2024-08-17T02:22:42.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 Processor -- start 2024-08-17T02:22:42.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.724+0000 7f6353fff700 1 -- start start 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6353fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6354067030 0x7f6354168880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6353fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63540cb5c0 0x7f6354169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6353fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cfb00 0x7f635415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6353fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540caee0 con 0x7f63540cfb00 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6353fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6354158a90 con 0x7f63540cb5c0 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6353fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6354168dc0 con 0x7f6354067030 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cfb00 0x7f635415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cfb00 0x7f635415a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50394/0 (socket says 172.21.15.100:50394) 2024-08-17T02:22:42.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 -- 172.21.15.100:0/1720745185 learned_addr learned my addr 172.21.15.100:0/1720745185 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6352ffd700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6354067030 0x7f6354168880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63527fc700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63540cb5c0 0x7f6354169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 -- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6354067030 msgr2=0x7f6354168880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6354067030 0x7f6354168880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 -- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63540cb5c0 msgr2=0x7f6354169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63540cb5c0 0x7f6354169980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 -- 172.21.15.100:0/1720745185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c009ae0 con 0x7f63540cfb00 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f63537fe700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cfb00 0x7f635415a3e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f634c009880 tx=0x7f634c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f633affd700 1 -- 172.21.15.100:0/1720745185 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c008b10 con 0x7f63540cfb00 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f633affd700 1 -- 172.21.15.100:0/1720745185 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c008c90 con 0x7f63540cfb00 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6354169ec0 con 0x7f63540cfb00 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f633affd700 1 -- 172.21.15.100:0/1720745185 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c0266b0 con 0x7f63540cfb00 2024-08-17T02:22:42.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.728+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635415cb30 con 0x7f63540cfb00 2024-08-17T02:22:42.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.732+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63540638f0 con 0x7f63540cfb00 2024-08-17T02:22:42.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.732+0000 7f633affd700 1 -- 172.21.15.100:0/1720745185 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f634c003040 con 0x7f63540cfb00 2024-08-17T02:22:42.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.732+0000 7f633affd700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6340060860 0x7f6340062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:42.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.732+0000 7f633affd700 1 -- 172.21.15.100:0/1720745185 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f634c07eb00 con 0x7f63540cfb00 2024-08-17T02:22:42.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.732+0000 7f6352ffd700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6340060860 0x7f6340062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:42.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.732+0000 7f6352ffd700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6340060860 0x7f6340062d10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6348009db0 tx=0x7f63480091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:42.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.736+0000 7f633affd700 1 -- 172.21.15.100:0/1720745185 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f634c04ef00 con 0x7f63540cfb00 2024-08-17T02:22:42.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:42.940+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f63540c6e00 con 0x7f63540cfb00 2024-08-17T02:22:43.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.604+0000 7f633affd700 1 -- 172.21.15.100:0/1720745185 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f634c052520 con 0x7f63540cfb00 2024-08-17T02:22:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6340060860 msgr2=0x7f6340062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6340060860 0x7f6340062d10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6348009db0 tx=0x7f63480091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cfb00 msgr2=0x7f635415a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cfb00 0x7f635415a3e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f634c009880 tx=0x7f634c018000 comp rx=0 tx=0).stop 2024-08-17T02:22:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 shutdown_connections 2024-08-17T02:22:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6340060860 0x7f6340062d10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cfb00 0x7f635415a3e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63540cb5c0 0x7f6354169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 --2- 172.21.15.100:0/1720745185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6354067030 0x7f6354168880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 >> 172.21.15.100:0/1720745185 conn(0x7f63540d0b50 msgr2=0x7f63540b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 shutdown_connections 2024-08-17T02:22:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.612+0000 7f6353fff700 1 -- 172.21.15.100:0/1720745185 wait complete. 2024-08-17T02:22:43.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-17T02:22:43.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecebfe700 1 Processor -- start 2024-08-17T02:22:43.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecebfe700 1 -- start start 2024-08-17T02:22:43.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecebfe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ec8159060 0x7f8ec80af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:43.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecebfe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159430 0x7f8ec80ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:43.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecebfe700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec815a640 con 0x7f8ec8159880 2024-08-17T02:22:43.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecebfe700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec80af7c0 con 0x7f8ec8159060 2024-08-17T02:22:43.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecebfe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec8159da0 con 0x7f8ec8159430 2024-08-17T02:22:43.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecdbfc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ec8159060 0x7f8ec80af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:43.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecd3fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159430 0x7f8ec80ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:43.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecd3fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159430 0x7f8ec80ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:49386/0 (socket says 172.21.15.100:49386) 2024-08-17T02:22:43.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecdbfc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ec8159060 0x7f8ec80af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:41852/0 (socket says 172.21.15.100:41852) 2024-08-17T02:22:43.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ecd3fb700 1 -- 172.21.15.100:0/3788765710 learned_addr learned my addr 172.21.15.100:0/3788765710 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:43.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.756+0000 7f8ece3fd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8ec8159880 0x7f8ec815cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:58352/0 (socket says 172.21.15.100:58352) 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecd3fb700 1 -- 172.21.15.100:0/3788765710 >> v2:172.21.15.123:3300/0 conn(0x7f8ec8159060 msgr2=0x7f8ec80af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecd3fb700 1 --2- 172.21.15.100:0/3788765710 >> v2:172.21.15.123:3300/0 conn(0x7f8ec8159060 0x7f8ec80af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecd3fb700 1 -- 172.21.15.100:0/3788765710 >> v1:172.21.15.123:6789/0 conn(0x7f8ec8159880 legacy=0x7f8ec815cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecd3fb700 1 -- 172.21.15.100:0/3788765710 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec815a080 con 0x7f8ec8159430 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecd3fb700 1 --2- 172.21.15.100:0/3788765710 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159430 0x7f8ec80ae020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099b0 tx=0x7f8ec4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa9c3abee33c8b69 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8eccbfa700 1 -- 172.21.15.100:0/3788765710 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4009ea0 con 0x7f8ec8159430 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8eccbfa700 1 -- 172.21.15.100:0/3788765710 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec400b860 con 0x7f8ec8159430 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8eccbfa700 1 -- 172.21.15.100:0/3788765710 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400bc00 con 0x7f8ec8159430 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecebfe700 1 -- 172.21.15.100:0/3788765710 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159430 msgr2=0x7f8ec80ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/3788765710 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159430 0x7f8ec80ae020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099b0 tx=0x7f8ec4009250 comp rx=0 tx=0).stop 2024-08-17T02:22:43.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecebfe700 1 -- 172.21.15.100:0/3788765710 shutdown_connections 2024-08-17T02:22:43.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/3788765710 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159430 0x7f8ec80ae020 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/3788765710 >> v2:172.21.15.123:3300/0 conn(0x7f8ec8159060 0x7f8ec80af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecebfe700 1 -- 172.21.15.100:0/3788765710 >> 172.21.15.100:0/3788765710 conn(0x7f8ec80a9d30 msgr2=0x7f8ec80ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:43.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecebfe700 1 -- 172.21.15.100:0/3788765710 shutdown_connections 2024-08-17T02:22:43.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.760+0000 7f8ecebfe700 1 -- 172.21.15.100:0/3788765710 wait complete. 2024-08-17T02:22:43.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 Processor -- start 2024-08-17T02:22:43.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 -- start start 2024-08-17T02:22:43.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159060 0x7f8ec80ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ec8159430 0x7f8ec816cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec8159880 0x7f8ec815ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec816d110 con 0x7f8ec8159880 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec816d340 con 0x7f8ec8159430 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec816d620 con 0x7f8ec8159060 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecd3fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ec8159430 0x7f8ec816cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ece3fd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec8159880 0x7f8ec815ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159060 0x7f8ec80ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159060 0x7f8ec80ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:49392/0 (socket says 172.21.15.100:49392) 2024-08-17T02:22:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 -- 172.21.15.100:0/2107113645 learned_addr learned my addr 172.21.15.100:0/2107113645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:43.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 -- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ec8159430 msgr2=0x7f8ec816cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:43.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ec8159430 0x7f8ec816cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 -- 172.21.15.100:0/2107113645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec8159880 msgr2=0x7f8ec815ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec8159880 0x7f8ec815ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 -- 172.21.15.100:0/2107113645 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40094f0 con 0x7f8ec8159060 2024-08-17T02:22:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecdbfc700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159060 0x7f8ec80ac000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800ec80 tx=0x7f8eb80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ebdffb700 1 -- 172.21.15.100:0/2107113645 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800c850 con 0x7f8ec8159060 2024-08-17T02:22:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ebdffb700 1 -- 172.21.15.100:0/2107113645 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb800c9b0 con 0x7f8ec8159060 2024-08-17T02:22:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ebdffb700 1 -- 172.21.15.100:0/2107113645 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800cd50 con 0x7f8ec8159060 2024-08-17T02:22:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec816d900 con 0x7f8ec8159060 2024-08-17T02:22:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.764+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec815bf70 con 0x7f8ec8159060 2024-08-17T02:22:43.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.768+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec80aa100 con 0x7f8ec8159060 2024-08-17T02:22:43.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.768+0000 7f8ebdffb700 1 -- 172.21.15.100:0/2107113645 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8eb8005a00 con 0x7f8ec8159060 2024-08-17T02:22:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.768+0000 7f8ebdffb700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8eac060860 0x7f8eac062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.768+0000 7f8ebdffb700 1 -- 172.21.15.100:0/2107113645 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f8eb807cff0 con 0x7f8ec8159060 2024-08-17T02:22:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.768+0000 7f8ecd3fb700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8eac060860 0x7f8eac062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:43.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.772+0000 7f8ecd3fb700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8eac060860 0x7f8eac062d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099b0 tx=0x7f8ec4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:43.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.776+0000 7f8ebdffb700 1 -- 172.21.15.100:0/2107113645 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8eb804d430 con 0x7f8ec8159060 2024-08-17T02:22:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:43.940+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ec80aa100 con 0x7f8ec8159060 2024-08-17T02:22:44.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.612+0000 7f8ebdffb700 1 -- 172.21.15.100:0/2107113645 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ec80aa100 con 0x7f8ec8159060 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8eac060860 msgr2=0x7f8eac062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8eac060860 0x7f8eac062d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099b0 tx=0x7f8ec4017000 comp rx=0 tx=0).stop 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159060 msgr2=0x7f8ec80ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159060 0x7f8ec80ac000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800ec80 tx=0x7f8eb80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 shutdown_connections 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8eac060860 0x7f8eac062d10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec8159880 0x7f8ec815ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ec8159430 0x7f8ec816cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 --2- 172.21.15.100:0/2107113645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ec8159060 0x7f8ec80ac000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 >> 172.21.15.100:0/2107113645 conn(0x7f8ec80a9d30 msgr2=0x7f8ec81569f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:44.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 shutdown_connections 2024-08-17T02:22:44.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.620+0000 7f8ecebfe700 1 -- 172.21.15.100:0/2107113645 wait complete. 2024-08-17T02:22:44.623 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-08-17T02:22:44.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-17T02:22:44.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.740+0000 7fd009fa0700 1 Processor -- start 2024-08-17T02:22:44.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.740+0000 7fd009fa0700 1 -- start start 2024-08-17T02:22:44.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd009fa0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 0x7fd0040b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:44.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd009fa0700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0040c5bb0 con 0x7fd0040cc400 2024-08-17T02:22:44.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd009fa0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0040c4fa0 con 0x7fd0040cea10 2024-08-17T02:22:44.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd009fa0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040c5280 con 0x7fd0040ce640 2024-08-17T02:22:44.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd00979f700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd0040cc400 0x7fd0040c6040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48870/0 (socket says 172.21.15.100:48870) 2024-08-17T02:22:44.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd00979f700 1 -- 172.21.15.100:0/782366802 learned_addr learned my addr 172.21.15.100:0/782366802 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:44.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd008f9e700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 0x7fd0040b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555101180 0 0) 0x7fd0040c5bb0 con 0x7fd0040cc400 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcff0003570 con 0x7fd0040cc400 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300766723 0 0) 0x7fd0040c4fa0 con 0x7fd0040cea10 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0040c5bb0 con 0x7fd0040cea10 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1023193579 0 0) 0x7fcff0003570 con 0x7fd0040cc400 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 msgr2=0x7fd0040b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 0x7fd0040b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 >> v1:172.21.15.123:6789/0 conn(0x7fd0040cea10 legacy=0x7fd0040cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040c5560 con 0x7fd0040cc400 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7fcffc0017f0 con 0x7fd0040cc400 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 >> v1:172.21.15.100:6789/0 conn(0x7fd0040cc400 legacy=0x7fd0040c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:44.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0003c20 0x7fcff0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fcff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 0x7fcff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040c5280 con 0x7fcff0004810 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff000bb40 con 0x7fcff0003c20 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff000bd00 con 0x7fd0040ce640 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd003fff700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fcff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd00979f700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 0x7fcff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd008f9e700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0003c20 0x7fcff0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd003fff700 1 -- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 msgr2=0x7fcff00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:44.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd003fff700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 0x7fcff00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd003fff700 1 -- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0003c20 msgr2=0x7fcff0004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd003fff700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0003c20 0x7fcff0004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd003fff700 1 -- 172.21.15.100:0/782366802 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff0003570 con 0x7fcff0004810 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd00979f700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040ce640 0x7fcff00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd003fff700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fcff0004c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcff4001a10 tx=0x7fcff4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db98ad519ad68172 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff400c5d0 con 0x7fcff0004810 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff400c730 con 0x7fcff0004810 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd0037fe700 1 -- 172.21.15.100:0/782366802 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff400cad0 con 0x7fcff0004810 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd009fa0700 1 -- 172.21.15.100:0/782366802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 msgr2=0x7fcff0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd009fa0700 1 --2- 172.21.15.100:0/782366802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fcff0004c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcff4001a10 tx=0x7fcff4007d30 comp rx=0 tx=0).stop 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd00979f700 1 -- 172.21.15.100:0/782366802 reap_dead start 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd009fa0700 1 -- 172.21.15.100:0/782366802 shutdown_connections 2024-08-17T02:22:44.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.744+0000 7fd009fa0700 1 -- 172.21.15.100:0/782366802 >> 172.21.15.100:0/782366802 conn(0x7fd004099410 msgr2=0x7fd0040675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:44.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 -- 172.21.15.100:0/782366802 shutdown_connections 2024-08-17T02:22:44.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 -- 172.21.15.100:0/782366802 wait complete. 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 Processor -- start 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 -- start start 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fd00409c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd00409c830 0x7fd0040c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040c8b20 0x7fd00416a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040ace20 con 0x7fcff0004810 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040c8150 con 0x7fd00409c830 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd009fa0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040c7540 con 0x7fd0040c8b20 2024-08-17T02:22:44.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd008f9e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fd00409c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd003fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd00409c830 0x7fd0040c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.748+0000 7fd008f9e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fd00409c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38354/0 (socket says 172.21.15.100:38354) 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd008f9e700 1 -- 172.21.15.100:0/452411482 learned_addr learned my addr 172.21.15.100:0/452411482 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd00979f700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040c8b20 0x7fd00416a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd008f9e700 1 -- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040c8b20 msgr2=0x7fd00416a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd008f9e700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040c8b20 0x7fd00416a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd008f9e700 1 -- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd00409c830 msgr2=0x7fd0040c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd008f9e700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd00409c830 0x7fd0040c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd008f9e700 1 -- 172.21.15.100:0/452411482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040c7820 con 0x7fcff0004810 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd008f9e700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fd00409c2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcfec011990 tx=0x7fcfec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd000ff9700 1 -- 172.21.15.100:0/452411482 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec015670 con 0x7fcff0004810 2024-08-17T02:22:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd000ff9700 1 -- 172.21.15.100:0/452411482 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec0157d0 con 0x7fcff0004810 2024-08-17T02:22:44.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0040c7b00 con 0x7fcff0004810 2024-08-17T02:22:44.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd000ff9700 1 -- 172.21.15.100:0/452411482 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec015b90 con 0x7fcff0004810 2024-08-17T02:22:44.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00416a730 con 0x7fcff0004810 2024-08-17T02:22:44.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0040c5d60 con 0x7fcff0004810 2024-08-17T02:22:44.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.752+0000 7fd000ff9700 1 -- 172.21.15.100:0/452411482 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcfec00f480 con 0x7fcff0004810 2024-08-17T02:22:44.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.756+0000 7fd000ff9700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfe4060860 0x7fcfe4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:44.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.756+0000 7fd000ff9700 1 -- 172.21.15.100:0/452411482 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fcfec07d1f0 con 0x7fcff0004810 2024-08-17T02:22:44.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.756+0000 7fd003fff700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfe4060860 0x7fcfe4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.756+0000 7fd003fff700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfe4060860 0x7fcfe4062d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcff4008fd0 tx=0x7fcff4015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:44.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.760+0000 7fd000ff9700 1 -- 172.21.15.100:0/452411482 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd0040c5d60 con 0x7fcff0004810 2024-08-17T02:22:44.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:44.924+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd0040c5d60 con 0x7fcff0004810 2024-08-17T02:22:45.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.620+0000 7fd000ff9700 1 -- 172.21.15.100:0/452411482 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd0040c5d60 con 0x7fcff0004810 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfe4060860 msgr2=0x7fcfe4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfe4060860 0x7fcfe4062d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcff4008fd0 tx=0x7fcff4015000 comp rx=0 tx=0).stop 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 msgr2=0x7fd00409c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fd00409c2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcfec011990 tx=0x7fcfec008040 comp rx=0 tx=0).stop 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 shutdown_connections 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfe4060860 0x7fcfe4062d10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0040c8b20 0x7fd00416a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd00409c830 0x7fd0040c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 --2- 172.21.15.100:0/452411482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcff0004810 0x7fd00409c2f0 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcfec011990 tx=0x7fcfec008040 comp rx=0 tx=0).stop 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 >> 172.21.15.100:0/452411482 conn(0x7fd004099410 msgr2=0x7fd004158790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:45.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 shutdown_connections 2024-08-17T02:22:45.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.624+0000 7fd009fa0700 1 -- 172.21.15.100:0/452411482 wait complete. 2024-08-17T02:22:45.628 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-08-17T02:22:45.642 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-17T02:22:45.642 INFO:tasks.ceph.mds.a:Stopping old one... 2024-08-17T02:22:45.642 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-17T02:22:45.643 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:22:45.643 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-17T02:22:45.643 INFO:tasks.ceph.mds.c:Stopping old one... 2024-08-17T02:22:45.643 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-17T02:22:45.643 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:22:45.643 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-17T02:22:45.643 INFO:tasks.ceph.mds.b:Stopping old one... 2024-08-17T02:22:45.644 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-17T02:22:45.644 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:22:45.644 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-17T02:22:45.644 INFO:tasks.ceph.mds.d:Stopping old one... 2024-08-17T02:22:45.644 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-17T02:22:45.644 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:22:45.654 INFO:tasks.ceph.mds.a:Stopped 2024-08-17T02:22:45.654 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-17T02:22:45.656 INFO:tasks.ceph.mds.b:Stopped 2024-08-17T02:22:45.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-17T02:22:45.656 INFO:tasks.ceph.mds.d:Stopped 2024-08-17T02:22:45.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-17T02:22:45.658 INFO:tasks.ceph.mds.c:Stopped 2024-08-17T02:22:45.658 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-17T02:22:45.659 INFO:tasks.ceph.mds.a:Started 2024-08-17T02:22:45.659 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:45.660 INFO:tasks.ceph.mds.c:Started 2024-08-17T02:22:45.660 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:45.700 INFO:tasks.ceph.mds.b:Started 2024-08-17T02:22:45.701 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:45.701 INFO:tasks.ceph.mds.d:Started 2024-08-17T02:22:45.701 DEBUG:teuthology.parallel:result is None 2024-08-17T02:22:45.702 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-08-17T02:22:45.730 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-08-17T02:22:45.735 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-08-17T02:22:45.777 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2024-08-17T02:22:45.779 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2024-08-17T02:22:45.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2a1a700 1 Processor -- start 2024-08-17T02:22:45.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2a1a700 1 -- start start 2024-08-17T02:22:45.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2a1a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f849c153320 0x7f849c1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2a1a700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f849c0cd1b0 0x7f849c15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2a1a700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849c15dbd0 con 0x7f849c15a190 2024-08-17T02:22:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2a1a700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849c0ce430 con 0x7f849c0cd1b0 2024-08-17T02:22:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2a1a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849c15d370 con 0x7f849c153320 2024-08-17T02:22:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2219700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f849c0cd1b0 0x7f849c15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a1217700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f849c15a190 0x7f849c0ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48880/0 (socket says 172.21.15.100:48880) 2024-08-17T02:22:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a1217700 1 -- 172.21.15.100:0/1316441457 learned_addr learned my addr 172.21.15.100:0/1316441457 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a1a18700 1 --2- 172.21.15.100:0/1316441457 >> v2:172.21.15.123:3300/0 conn(0x7f849c153320 0x7f849c1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2219700 1 -- 172.21.15.100:0/1316441457 >> v2:172.21.15.123:3300/0 conn(0x7f849c153320 msgr2=0x7f849c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2219700 1 --2- 172.21.15.100:0/1316441457 >> v2:172.21.15.123:3300/0 conn(0x7f849c153320 0x7f849c1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2219700 1 -- 172.21.15.100:0/1316441457 >> v1:172.21.15.100:6789/0 conn(0x7f849c15a190 legacy=0x7f849c0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2219700 1 -- 172.21.15.100:0/1316441457 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c15a590 con 0x7f849c0cd1b0 2024-08-17T02:22:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a0a16700 1 -- 172.21.15.100:0/1316441457 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24845946 0 0) 0x7f849c15dbd0 con 0x7f849c15a190 2024-08-17T02:22:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a2219700 1 --2- 172.21.15.100:0/1316441457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c15cc60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f849c0d0cd0 tx=0x7f848c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43716fb48b323d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a0a16700 1 -- 172.21.15.100:0/1316441457 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c011630 con 0x7f849c0cd1b0 2024-08-17T02:22:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a0a16700 1 -- 172.21.15.100:0/1316441457 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c011790 con 0x7f849c0cd1b0 2024-08-17T02:22:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.812+0000 7f84a0a16700 1 -- 172.21.15.100:0/1316441457 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c011b30 con 0x7f849c0cd1b0 2024-08-17T02:22:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 -- 172.21.15.100:0/1316441457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 msgr2=0x7f849c15cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/1316441457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c15cc60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f849c0d0cd0 tx=0x7f848c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 -- 172.21.15.100:0/1316441457 shutdown_connections 2024-08-17T02:22:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/1316441457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c15cc60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/1316441457 >> v2:172.21.15.123:3300/0 conn(0x7f849c153320 0x7f849c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 -- 172.21.15.100:0/1316441457 >> 172.21.15.100:0/1316441457 conn(0x7f849c0d00a0 msgr2=0x7f849c0d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 -- 172.21.15.100:0/1316441457 shutdown_connections 2024-08-17T02:22:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 -- 172.21.15.100:0/1316441457 wait complete. 2024-08-17T02:22:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 Processor -- start 2024-08-17T02:22:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 -- start start 2024-08-17T02:22:45.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.816+0000 7f84a2a1a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:45.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2a1a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c153320 0x7f849c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:45.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2a1a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c15a190 0x7f849c168bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:45.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2a1a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c0b4470 con 0x7f849c0cd1b0 2024-08-17T02:22:45.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2a1a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c0c80d0 con 0x7f849c153320 2024-08-17T02:22:45.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2a1a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849c0b0b60 con 0x7f849c15a190 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38394/0 (socket says 172.21.15.100:38394) 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 -- 172.21.15.100:0/3198370608 learned_addr learned my addr 172.21.15.100:0/3198370608 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1217700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c153320 0x7f849c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2219700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c15a190 0x7f849c168bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 -- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c15a190 msgr2=0x7f849c168bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c15a190 0x7f849c168bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 -- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c153320 msgr2=0x7f849c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c153320 0x7f849c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 -- 172.21.15.100:0/3198370608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009420 con 0x7f849c0cd1b0 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a1a18700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c0c7b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f849000ec80 tx=0x7f84900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f8499ffb700 1 -- 172.21.15.100:0/3198370608 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849000c9d0 con 0x7f849c0cd1b0 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f8499ffb700 1 -- 172.21.15.100:0/3198370608 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849000cb30 con 0x7f849c0cd1b0 2024-08-17T02:22:45.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849c0b0de0 con 0x7f849c0cd1b0 2024-08-17T02:22:45.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f8499ffb700 1 -- 172.21.15.100:0/3198370608 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490013470 con 0x7f849c0cd1b0 2024-08-17T02:22:45.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c0b13a0 con 0x7f849c0cd1b0 2024-08-17T02:22:45.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.820+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849c0b15c0 con 0x7f849c0cd1b0 2024-08-17T02:22:45.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.824+0000 7f8499ffb700 1 -- 172.21.15.100:0/3198370608 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8490013820 con 0x7f849c0cd1b0 2024-08-17T02:22:45.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.824+0000 7f8499ffb700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8488060860 0x7f8488062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.824+0000 7f8499ffb700 1 -- 172.21.15.100:0/3198370608 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f849007cc60 con 0x7f849c0cd1b0 2024-08-17T02:22:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.824+0000 7f84a1217700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8488060860 0x7f8488062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.824+0000 7f84a1217700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8488060860 0x7f8488062d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f849c0b3840 tx=0x7f8494009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:45.832+0000 7f8499ffb700 1 -- 172.21.15.100:0/3198370608 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f849004d310 con 0x7f849c0cd1b0 2024-08-17T02:22:46.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.004+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f849c0638f0 con 0x7f849c0cd1b0 2024-08-17T02:22:46.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.632+0000 7f8499ffb700 1 -- 172.21.15.100:0/3198370608 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8490050930 con 0x7f849c0cd1b0 2024-08-17T02:22:46.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8488060860 msgr2=0x7f8488062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:46.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8488060860 0x7f8488062d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f849c0b3840 tx=0x7f8494009040 comp rx=0 tx=0).stop 2024-08-17T02:22:46.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 msgr2=0x7f849c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:46.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c0c7b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f849000ec80 tx=0x7f84900091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 shutdown_connections 2024-08-17T02:22:46.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8488060860 0x7f8488062d10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c15a190 0x7f849c168bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c153320 0x7f849c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 --2- 172.21.15.100:0/3198370608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c0cd1b0 0x7f849c0c7b90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 >> 172.21.15.100:0/3198370608 conn(0x7f849c0d00a0 msgr2=0x7f849c066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:46.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 shutdown_connections 2024-08-17T02:22:46.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.636+0000 7f84a2a1a700 1 -- 172.21.15.100:0/3198370608 wait complete. 2024-08-17T02:22:46.638 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-08-17T02:22:46.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-08-17T02:22:46.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 Processor -- start 2024-08-17T02:22:46.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 -- start start 2024-08-17T02:22:46.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f48d8159040 0x7f48d80cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:46.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 0x7f48d80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:46.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f48d815cea0 0x7f48d815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:46.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d80cf190 con 0x7f48d815cea0 2024-08-17T02:22:46.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d80d09a0 con 0x7f48d8159040 2024-08-17T02:22:46.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80ce5e0 con 0x7f48d80d0010 2024-08-17T02:22:46.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f48d815cea0 0x7f48d815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:46.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f48d815cea0 0x7f48d815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38396/0 (socket says 172.21.15.100:38396) 2024-08-17T02:22:46.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 -- 172.21.15.100:0/3251201759 learned_addr learned my addr 172.21.15.100:0/3251201759 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:46.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d6ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 0x7f48d80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:46.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d77fe700 1 --2- 172.21.15.100:0/3251201759 >> v2:172.21.15.123:3300/0 conn(0x7f48d8159040 0x7f48d80cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 -- 172.21.15.100:0/3251201759 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 msgr2=0x7f48d80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 --2- 172.21.15.100:0/3251201759 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 0x7f48d80d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 -- 172.21.15.100:0/3251201759 >> v2:172.21.15.123:3300/0 conn(0x7f48d8159040 msgr2=0x7f48d80cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 --2- 172.21.15.100:0/3251201759 >> v2:172.21.15.123:3300/0 conn(0x7f48d8159040 0x7f48d80cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 -- 172.21.15.100:0/3251201759 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d80ce860 con 0x7f48d815cea0 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d7fff700 1 --2- 172.21.15.100:0/3251201759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d815d320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48d8070d70 tx=0x7f48d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b252f772009da10f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d67fc700 1 -- 172.21.15.100:0/3251201759 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d000e070 con 0x7f48d815cea0 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d67fc700 1 -- 172.21.15.100:0/3251201759 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d000b970 con 0x7f48d815cea0 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48d67fc700 1 -- 172.21.15.100:0/3251201759 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d00092e0 con 0x7f48d815cea0 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 -- 172.21.15.100:0/3251201759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 msgr2=0x7f48d815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.804+0000 7f48dca16700 1 --2- 172.21.15.100:0/3251201759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d815d320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48d8070d70 tx=0x7f48d0009840 comp rx=0 tx=0).stop 2024-08-17T02:22:46.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 -- 172.21.15.100:0/3251201759 shutdown_connections 2024-08-17T02:22:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 --2- 172.21.15.100:0/3251201759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d815d320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 --2- 172.21.15.100:0/3251201759 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 0x7f48d80d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 --2- 172.21.15.100:0/3251201759 >> v2:172.21.15.123:3300/0 conn(0x7f48d8159040 0x7f48d80cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 -- 172.21.15.100:0/3251201759 >> 172.21.15.100:0/3251201759 conn(0x7f48d8068250 msgr2=0x7f48d8068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 -- 172.21.15.100:0/3251201759 shutdown_connections 2024-08-17T02:22:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 -- 172.21.15.100:0/3251201759 wait complete. 2024-08-17T02:22:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 Processor -- start 2024-08-17T02:22:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 -- start start 2024-08-17T02:22:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 0x7f48d80b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48d8159040 0x7f48d80adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d8068620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80a9220 con 0x7f48d815cea0 2024-08-17T02:22:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80b1940 con 0x7f48d8159040 2024-08-17T02:22:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48dca16700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80a8610 con 0x7f48d80d0010 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d8068620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d77fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 0x7f48d80b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48d8159040 0x7f48d80adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d8068620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38410/0 (socket says 172.21.15.100:38410) 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d7fff700 1 -- 172.21.15.100:0/159344468 learned_addr learned my addr 172.21.15.100:0/159344468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d7fff700 1 -- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 msgr2=0x7f48d80b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d7fff700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 0x7f48d80b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d7fff700 1 -- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48d8159040 msgr2=0x7f48d80adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d7fff700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48d8159040 0x7f48d80adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.808+0000 7f48d7fff700 1 -- 172.21.15.100:0/159344468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0009ae0 con 0x7f48d815cea0 2024-08-17T02:22:46.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48d7fff700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d8068620 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f48d000b610 tx=0x7f48d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:46.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48bf7fe700 1 -- 172.21.15.100:0/159344468 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0008b00 con 0x7f48d815cea0 2024-08-17T02:22:46.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48bf7fe700 1 -- 172.21.15.100:0/159344468 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d0008c80 con 0x7f48d815cea0 2024-08-17T02:22:46.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48bf7fe700 1 -- 172.21.15.100:0/159344468 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0026460 con 0x7f48d815cea0 2024-08-17T02:22:46.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d80a88f0 con 0x7f48d815cea0 2024-08-17T02:22:46.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d80a8eb0 con 0x7f48d815cea0 2024-08-17T02:22:46.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d80643a0 con 0x7f48d815cea0 2024-08-17T02:22:46.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48bf7fe700 1 -- 172.21.15.100:0/159344468 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f48d0003040 con 0x7f48d815cea0 2024-08-17T02:22:46.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48bf7fe700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f48c4060860 0x7f48c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:46.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48bf7fe700 1 -- 172.21.15.100:0/159344468 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f48d007e5a0 con 0x7f48d815cea0 2024-08-17T02:22:46.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.812+0000 7f48d77fe700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f48c4060860 0x7f48c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:46.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.816+0000 7f48d77fe700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f48c4060860 0x7f48c4062d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009db0 tx=0x7f48c80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:46.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:46.820+0000 7f48bf7fe700 1 -- 172.21.15.100:0/159344468 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f48d004ed40 con 0x7f48d815cea0 2024-08-17T02:22:47.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f48d80cfdf0 con 0x7f48d815cea0 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48bf7fe700 1 -- 172.21.15.100:0/159344468 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f48d0052360 con 0x7f48d815cea0 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAVCcBmoa7GLBAAEa267zm/LtpX8rM4H7yg8w==", 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.046 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAVCcBmlTGVNhAAQ0osukqK1NPnmS0+QVksnA==", 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.047 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAVCcBmF/fPMhAAfa+QY0oBhYx9nTuka/+TdA==", 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.048 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAVCcBmxqYBOhAA0HeZWzfPBTtfFOpQaxW/Hw==", 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAYCcBmgnZCNhAA1nISJnYxJYmnry99/RzJgw==", 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.049 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAbCcBmvSDFOhAAK0vSqY2GBGTvVn4FKXaIfA==", 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAfCcBmQ/wSAxAAvRzMbFLmZWCyN6aUkIF6NA==", 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAiCcBmc5FsBRAAldQsOEVDCiN/naewTHmTZQ==", 2024-08-17T02:22:47.051 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAnCcBmNiSCKBAAB8xY3SkplnjexHfk3MGyGw==", 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqCcBmknnQKhAA+i/GGSIpkfT001Fnt6aSJg==", 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtCcBmocZgLxAAejHqqtvypf2E+OBjcKNS6A==", 2024-08-17T02:22:47.053 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwCcBmPHC+MxAAop04f8WqhE1Ovi2r7HhxZw==", 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.054 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAWCcBmxiJDBBAAOJ7HO5oxzLmoqqDQJJIT1w==", 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.055 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAWCcBmBW+sDBAAdA1JhVXsr5HnAiu/iIxSRg==", 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAWCcBmZOmsCRAAflJ9DnDrX79J5lpCR+YRSA==", 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-08-17T02:22:47.056 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAWCcBmkOhsDxAAqAgtA04spf0ioafXteRTSg==", 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-08-17T02:22:47.057 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAVCcBmWDSeGhAA1ntawk4QYXQDVdc0fKou/A==", 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.058 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-mds", 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6CcBmLPlPOxAAl9m+47pcKL8r1eJ2Lnhtqg==", 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-mds" 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-mgr", 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6CcBmMxNQOxAAcSWD6Hs5X+4ph5bpQ9u9FQ==", 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-mgr" 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-osd", 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6CcBmCyxQOxAAo0XSLM4H9P8weqJCx1Gnzw==", 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-osd" 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rbd", 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6CcBm1EJQOxAAxTpMEUaatiVK7gXbQmvAkA==", 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rbd" 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6CcBmXllQOxAADjD9ml9hcreS+ZKihY1BrA==", 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rgw", 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6CcBmdXFQOxAAk0tnKc8ZjSlhqE3eQUkk0Q==", 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rgw" 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAVCcBmxt72KRAAwupnsLYFrW5/0eK7BF9Ecg==", 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAVCcBm2PEUJxAAoMsk1QkbyySm/5QCLvxh9A==", 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-08-17T02:22:47.062 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-08-17T02:22:47.063 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-08-17T02:22:47.063 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-08-17T02:22:47.063 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.063 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:47.063 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-08-17T02:22:47.063 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:22:47.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f48c4060860 msgr2=0x7f48c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f48c4060860 0x7f48c4062d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009db0 tx=0x7f48c80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 msgr2=0x7f48d8068620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d8068620 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f48d000b610 tx=0x7f48d0018000 comp rx=0 tx=0).stop 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 shutdown_connections 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f48c4060860 0x7f48c4062d10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48d815cea0 0x7f48d8068620 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48d8159040 0x7f48d80adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 --2- 172.21.15.100:0/159344468 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48d80d0010 0x7f48d80b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 >> 172.21.15.100:0/159344468 conn(0x7f48d8068250 msgr2=0x7f48d8154070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 shutdown_connections 2024-08-17T02:22:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.044+0000 7f48dca16700 1 -- 172.21.15.100:0/159344468 wait complete. 2024-08-17T02:22:47.066 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-08-17T02:22:47.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee5759e700 1 Processor -- start 2024-08-17T02:22:47.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee5759e700 1 -- start start 2024-08-17T02:22:47.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee5759e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fee580cb560 0x7fee58161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee5759e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee580cbaa0 con 0x7fee580b14a0 2024-08-17T02:22:47.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee5759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee58157630 con 0x7fee58154900 2024-08-17T02:22:47.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee5759e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee58068790 con 0x7fee580cb560 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee56d9d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fee580cb560 0x7fee58161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee55d9b700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fee580b14a0 0x7fee580cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48884/0 (socket says 172.21.15.100:48884) 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.176+0000 7fee55d9b700 1 -- 172.21.15.100:0/2019504970 learned_addr learned my addr 172.21.15.100:0/2019504970 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee56d9d700 1 -- 172.21.15.100:0/2019504970 >> v1:172.21.15.123:6789/0 conn(0x7fee58154900 legacy=0x7fee580b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee56d9d700 1 -- 172.21.15.100:0/2019504970 >> v1:172.21.15.100:6789/0 conn(0x7fee580b14a0 legacy=0x7fee580cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee56d9d700 1 -- 172.21.15.100:0/2019504970 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58157910 con 0x7fee580cb560 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5559a700 1 -- 172.21.15.100:0/2019504970 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859621336 0 0) 0x7fee580cbaa0 con 0x7fee580b14a0 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee56d9d700 1 --2- 172.21.15.100:0/2019504970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580cb560 0x7fee58161ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee50009e60 tx=0x7fee50009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ddbcd533bc4cec server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5559a700 1 -- 172.21.15.100:0/2019504970 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000e070 con 0x7fee580cb560 2024-08-17T02:22:47.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5559a700 1 -- 172.21.15.100:0/2019504970 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5000bc30 con 0x7fee580cb560 2024-08-17T02:22:47.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5559a700 1 -- 172.21.15.100:0/2019504970 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50012430 con 0x7fee580cb560 2024-08-17T02:22:47.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5759e700 1 -- 172.21.15.100:0/2019504970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580cb560 msgr2=0x7fee58161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5759e700 1 --2- 172.21.15.100:0/2019504970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580cb560 0x7fee58161ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee50009e60 tx=0x7fee50009780 comp rx=0 tx=0).stop 2024-08-17T02:22:47.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5759e700 1 -- 172.21.15.100:0/2019504970 shutdown_connections 2024-08-17T02:22:47.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5759e700 1 --2- 172.21.15.100:0/2019504970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580cb560 0x7fee58161ad0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5759e700 1 -- 172.21.15.100:0/2019504970 >> 172.21.15.100:0/2019504970 conn(0x7fee580678d0 msgr2=0x7fee58067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:47.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5759e700 1 -- 172.21.15.100:0/2019504970 shutdown_connections 2024-08-17T02:22:47.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.180+0000 7fee5759e700 1 -- 172.21.15.100:0/2019504970 wait complete. 2024-08-17T02:22:47.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 Processor -- start 2024-08-17T02:22:47.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 -- start start 2024-08-17T02:22:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580b14a0 0x7fee58156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee580cb560 0x7fee58142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee58154900 0x7fee58159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58156700 con 0x7fee580b14a0 2024-08-17T02:22:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58151040 con 0x7fee580cb560 2024-08-17T02:22:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5815a170 con 0x7fee58154900 2024-08-17T02:22:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580b14a0 0x7fee58156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580b14a0 0x7fee58156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38426/0 (socket says 172.21.15.100:38426) 2024-08-17T02:22:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 -- 172.21.15.100:0/2142520768 learned_addr learned my addr 172.21.15.100:0/2142520768 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee56d9d700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee58154900 0x7fee58159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee55d9b700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee580cb560 0x7fee58142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 -- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee58154900 msgr2=0x7fee58159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee58154900 0x7fee58159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 -- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee580cb560 msgr2=0x7fee58142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee580cb560 0x7fee58142660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 -- 172.21.15.100:0/2142520768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee50009a20 con 0x7fee580b14a0 2024-08-17T02:22:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5659c700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580b14a0 0x7fee58156b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c4e0 tx=0x7fee4800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee4e7fc700 1 -- 172.21.15.100:0/2142520768 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48011590 con 0x7fee580b14a0 2024-08-17T02:22:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee4e7fc700 1 -- 172.21.15.100:0/2142520768 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee48011710 con 0x7fee580b14a0 2024-08-17T02:22:47.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee4e7fc700 1 -- 172.21.15.100:0/2142520768 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48011ad0 con 0x7fee580b14a0 2024-08-17T02:22:47.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.184+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee581570d0 con 0x7fee580b14a0 2024-08-17T02:22:47.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.188+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581560b0 con 0x7fee580b14a0 2024-08-17T02:22:47.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.188+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee580643a0 con 0x7fee580b14a0 2024-08-17T02:22:47.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.188+0000 7fee4e7fc700 1 -- 172.21.15.100:0/2142520768 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fee4801b3b0 con 0x7fee580b14a0 2024-08-17T02:22:47.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.188+0000 7fee4e7fc700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fee38060860 0x7fee38062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.188+0000 7fee4e7fc700 1 -- 172.21.15.100:0/2142520768 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fee4807bc70 con 0x7fee580b14a0 2024-08-17T02:22:47.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.188+0000 7fee55d9b700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fee38060860 0x7fee38062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.188+0000 7fee55d9b700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fee38060860 0x7fee38062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee58153a20 tx=0x7fee40007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.196+0000 7fee4e7fc700 1 -- 172.21.15.100:0/2142520768 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fee4804c410 con 0x7fee580b14a0 2024-08-17T02:22:47.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.416+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fee58063100 con 0x7fee580b14a0 2024-08-17T02:22:47.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.424+0000 7fee4e7fc700 1 -- 172.21.15.100:0/2142520768 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fee48017070 con 0x7fee580b14a0 2024-08-17T02:22:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fee38060860 msgr2=0x7fee38062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fee38060860 0x7fee38062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee58153a20 tx=0x7fee40007d30 comp rx=0 tx=0).stop 2024-08-17T02:22:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580b14a0 msgr2=0x7fee58156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580b14a0 0x7fee58156b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c4e0 tx=0x7fee4800c7f0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 shutdown_connections 2024-08-17T02:22:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fee38060860 0x7fee38062d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee58154900 0x7fee58159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee580cb560 0x7fee58142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 --2- 172.21.15.100:0/2142520768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580b14a0 0x7fee58156b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 >> 172.21.15.100:0/2142520768 conn(0x7fee580678d0 msgr2=0x7fee580aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 shutdown_connections 2024-08-17T02:22:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.428+0000 7fee5759e700 1 -- 172.21.15.100:0/2142520768 wait complete. 2024-08-17T02:22:47.442 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-08-17T02:22:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9daa606700 1 Processor -- start 2024-08-17T02:22:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9daa606700 1 -- start start 2024-08-17T02:22:47.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9daa606700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9da4155d20 0x7f9da40b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9daa606700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da406df10 0x7f9da406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9daa606700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da406e8b0 con 0x7f9da40ad7c0 2024-08-17T02:22:47.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9daa606700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da40adc10 con 0x7f9da4155d20 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9daa606700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40af2b0 con 0x7f9da406df10 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9da8e03700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9da40ad7c0 0x7f9da40aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48892/0 (socket says 172.21.15.100:48892) 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9da8e03700 1 -- 172.21.15.100:0/884002559 learned_addr learned my addr 172.21.15.100:0/884002559 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9da9604700 1 --2- 172.21.15.100:0/884002559 >> v2:172.21.15.123:3300/0 conn(0x7f9da4155d20 0x7f9da40b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9da9e05700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da406df10 0x7f9da406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.548+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928972086 0 0) 0x7f9d8c001110 con 0x7f9da40ad7c0 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d90005780 con 0x7f9da40ad7c0 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1730887976 0 0) 0x7f9d90005780 con 0x7f9da40ad7c0 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da406df10 msgr2=0x7f9da406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da406df10 0x7f9da406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 >> v2:172.21.15.123:3300/0 conn(0x7f9da4155d20 msgr2=0x7f9da40b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 --2- 172.21.15.100:0/884002559 >> v2:172.21.15.123:3300/0 conn(0x7f9da4155d20 0x7f9da40b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40af590 con 0x7f9da40ad7c0 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f9d8c001600 con 0x7f9da40ad7c0 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 >> v1:172.21.15.100:6789/0 conn(0x7f9da40ad7c0 legacy=0x7f9da40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90005c70 0x7f9d90006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9d90006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da406df10 0x7f9d900073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40adc10 con 0x7f9d90006a20 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40af2b0 con 0x7f9d90005c70 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9000dc70 con 0x7f9da406df10 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da8e03700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9d90006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da9604700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90005c70 0x7f9d90006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da9e05700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da406df10 0x7f9d900073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da8e03700 1 -- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da406df10 msgr2=0x7f9d900073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da8e03700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da406df10 0x7f9d900073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da8e03700 1 -- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90005c70 msgr2=0x7f9d90006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da8e03700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90005c70 0x7f9d90006080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da8e03700 1 -- 172.21.15.100:0/884002559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d90005780 con 0x7f9d90006a20 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da9604700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90005c70 0x7f9d90006080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da8e03700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9d90006e90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c620 tx=0x7f9d8c007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30ab4d6fba5b41d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00cde0 con 0x7f9d90006a20 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c005920 con 0x7f9d90006a20 2024-08-17T02:22:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da3fff700 1 -- 172.21.15.100:0/884002559 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c005ce0 con 0x7f9d90006a20 2024-08-17T02:22:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9daa606700 1 -- 172.21.15.100:0/884002559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 msgr2=0x7f9d90006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9daa606700 1 --2- 172.21.15.100:0/884002559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9d90006e90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c620 tx=0x7f9d8c007a80 comp rx=0 tx=0).stop 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9da9e05700 1 -- 172.21.15.100:0/884002559 reap_dead start 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9daa606700 1 -- 172.21.15.100:0/884002559 shutdown_connections 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9daa606700 1 -- 172.21.15.100:0/884002559 >> 172.21.15.100:0/884002559 conn(0x7f9da40678d0 msgr2=0x7f9da4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9daa606700 1 -- 172.21.15.100:0/884002559 shutdown_connections 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9daa606700 1 -- 172.21.15.100:0/884002559 wait complete. 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9daa606700 1 Processor -- start 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.552+0000 7f9daa606700 1 -- start start 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9da4145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da41478e0 0x7f9da414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9da414d2a0 0x7f9da414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da41480c0 con 0x7f9d90006a20 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da4157c20 con 0x7f9da414d2a0 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da414a520 con 0x7f9da41478e0 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9da4145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9da4145ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38438/0 (socket says 172.21.15.100:38438) 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da8e03700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da41478e0 0x7f9da414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 -- 172.21.15.100:0/3942681977 learned_addr learned my addr 172.21.15.100:0/3942681977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9e05700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9da414d2a0 0x7f9da414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 -- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da41478e0 msgr2=0x7f9da414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da41478e0 0x7f9da414ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 -- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9da414d2a0 msgr2=0x7f9da414efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9da414d2a0 0x7f9da414efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 -- 172.21.15.100:0/3942681977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da4147de0 con 0x7f9d90006a20 2024-08-17T02:22:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da9604700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9da4145ac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9d94010570 tx=0x7f9d9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da17fa700 1 -- 172.21.15.100:0/3942681977 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400e050 con 0x7f9d90006a20 2024-08-17T02:22:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da17fa700 1 -- 172.21.15.100:0/3942681977 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9400f040 con 0x7f9d90006a20 2024-08-17T02:22:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da41575d0 con 0x7f9d90006a20 2024-08-17T02:22:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9da17fa700 1 -- 172.21.15.100:0/3942681977 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94015670 con 0x7f9d90006a20 2024-08-17T02:22:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da41580b0 con 0x7f9d90006a20 2024-08-17T02:22:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.556+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da4157a50 con 0x7f9d90006a20 2024-08-17T02:22:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.560+0000 7f9da17fa700 1 -- 172.21.15.100:0/3942681977 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9d94003020 con 0x7f9d90006a20 2024-08-17T02:22:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.560+0000 7f9da17fa700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d84060880 0x7f9d84062d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.560+0000 7f9da17fa700 1 -- 172.21.15.100:0/3942681977 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9d9407d3a0 con 0x7f9d90006a20 2024-08-17T02:22:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.560+0000 7f9da8e03700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d84060880 0x7f9d84062d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.560+0000 7f9da8e03700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d84060880 0x7f9d84062d30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c540 tx=0x7f9d8c015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.568+0000 7f9da17fa700 1 -- 172.21.15.100:0/3942681977 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9d9404daa0 con 0x7f9d90006a20 2024-08-17T02:22:47.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.792+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9da40643a0 con 0x7f9d90006a20 2024-08-17T02:22:47.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.796+0000 7f9da17fa700 1 -- 172.21.15.100:0/3942681977 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9d940510c0 con 0x7f9d90006a20 2024-08-17T02:22:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d84060880 msgr2=0x7f9d84062d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d84060880 0x7f9d84062d30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c540 tx=0x7f9d8c015000 comp rx=0 tx=0).stop 2024-08-17T02:22:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 msgr2=0x7f9da4145ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9da4145ac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9d94010570 tx=0x7f9d9400b040 comp rx=0 tx=0).stop 2024-08-17T02:22:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 shutdown_connections 2024-08-17T02:22:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9d84060880 0x7f9d84062d30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9da414d2a0 0x7f9da414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9da41478e0 0x7f9da414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 --2- 172.21.15.100:0/3942681977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d90006a20 0x7f9da4145ac0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 >> 172.21.15.100:0/3942681977 conn(0x7f9da40678d0 msgr2=0x7f9da41467c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 shutdown_connections 2024-08-17T02:22:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.800+0000 7f9daa606700 1 -- 172.21.15.100:0/3942681977 wait complete. 2024-08-17T02:22:47.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-08-17T02:22:47.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.924+0000 7f469f322700 1 Processor -- start 2024-08-17T02:22:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.924+0000 7f469f322700 1 -- start start 2024-08-17T02:22:47.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 0x7f46980aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f469806e620 0x7f46980ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46980ac000 con 0x7f46980aecc0 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46980ab3f0 con 0x7f469806e620 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46980ab6d0 con 0x7f4698158eb0 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469eb21700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f469806e620 0x7f46980ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469eb21700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f469806e620 0x7f46980ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38444/0 (socket says 172.21.15.100:38444) 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469eb21700 1 -- 172.21.15.100:0/80880876 learned_addr learned my addr 172.21.15.100:0/80880876 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469e320700 1 --2- 172.21.15.100:0/80880876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 0x7f46980aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469eb21700 1 -- 172.21.15.100:0/80880876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 msgr2=0x7f46980aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469eb21700 1 --2- 172.21.15.100:0/80880876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 0x7f46980aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469eb21700 1 -- 172.21.15.100:0/80880876 >> v1:172.21.15.123:6789/0 conn(0x7f46980aecc0 legacy=0x7f469806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469eb21700 1 -- 172.21.15.100:0/80880876 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980ab9b0 con 0x7f469806e620 2024-08-17T02:22:47.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469eb21700 1 --2- 172.21.15.100:0/80880876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f46980ac490 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46900099b0 tx=0x7f4690009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be3ccbe619f6b4c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469d31e700 1 -- 172.21.15.100:0/80880876 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469000b860 con 0x7f469806e620 2024-08-17T02:22:47.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469d31e700 1 -- 172.21.15.100:0/80880876 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469000b9c0 con 0x7f469806e620 2024-08-17T02:22:47.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469d31e700 1 -- 172.21.15.100:0/80880876 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469000bd60 con 0x7f469806e620 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 -- 172.21.15.100:0/80880876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 msgr2=0x7f46980ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 --2- 172.21.15.100:0/80880876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f46980ac490 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46900099b0 tx=0x7f4690009250 comp rx=0 tx=0).stop 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 -- 172.21.15.100:0/80880876 shutdown_connections 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 --2- 172.21.15.100:0/80880876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f46980ac490 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 --2- 172.21.15.100:0/80880876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 0x7f46980aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 -- 172.21.15.100:0/80880876 >> 172.21.15.100:0/80880876 conn(0x7f46980678d0 msgr2=0x7f4698067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 -- 172.21.15.100:0/80880876 shutdown_connections 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 -- 172.21.15.100:0/80880876 wait complete. 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.928+0000 7f469f322700 1 Processor -- start 2024-08-17T02:22:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 -- start start 2024-08-17T02:22:47.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f4698140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980aecc0 0x7f469815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 0x7f469815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4698140790 con 0x7f469806e620 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469815b790 con 0x7f46980aecc0 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469815ca10 con 0x7f4698158eb0 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f4698140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f4698140250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38450/0 (socket says 172.21.15.100:38450) 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 -- 172.21.15.100:0/1318106587 learned_addr learned my addr 172.21.15.100:0/1318106587 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469eb21700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 0x7f469815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469db1f700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980aecc0 0x7f469815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 -- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 msgr2=0x7f469815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 0x7f469815bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 -- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980aecc0 msgr2=0x7f469815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:47.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980aecc0 0x7f469815c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:47.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 -- 172.21.15.100:0/1318106587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46900094f0 con 0x7f469806e620 2024-08-17T02:22:47.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469e320700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f4698140250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f468c00d860 tx=0x7f468c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f468a7fc700 1 -- 172.21.15.100:0/1318106587 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c0125f0 con 0x7f469806e620 2024-08-17T02:22:47.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f468a7fc700 1 -- 172.21.15.100:0/1318106587 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c012770 con 0x7f469806e620 2024-08-17T02:22:47.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469815a060 con 0x7f469806e620 2024-08-17T02:22:47.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f468a7fc700 1 -- 172.21.15.100:0/1318106587 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c012b30 con 0x7f469806e620 2024-08-17T02:22:47.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.932+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469815b420 con 0x7f469806e620 2024-08-17T02:22:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.936+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46980643a0 con 0x7f469806e620 2024-08-17T02:22:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.936+0000 7f468a7fc700 1 -- 172.21.15.100:0/1318106587 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f468c013480 con 0x7f469806e620 2024-08-17T02:22:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.936+0000 7f468a7fc700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f467c060860 0x7f467c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.936+0000 7f468a7fc700 1 -- 172.21.15.100:0/1318106587 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f468c07cbf0 con 0x7f469806e620 2024-08-17T02:22:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.936+0000 7f469db1f700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f467c060860 0x7f467c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.936+0000 7f469db1f700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f467c060860 0x7f467c062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f46980b0140 tx=0x7f46940016f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:47.944+0000 7f468a7fc700 1 -- 172.21.15.100:0/1318106587 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f468c04d390 con 0x7f469806e620 2024-08-17T02:22:48.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.160+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f469806f5f0 con 0x7f469806e620 2024-08-17T02:22:48.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f468a7fc700 1 -- 172.21.15.100:0/1318106587 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f468c0509b0 con 0x7f469806e620 2024-08-17T02:22:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f467c060860 msgr2=0x7f467c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f467c060860 0x7f467c062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f46980b0140 tx=0x7f46940016f0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 msgr2=0x7f4698140250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f4698140250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f468c00d860 tx=0x7f468c008040 comp rx=0 tx=0).stop 2024-08-17T02:22:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 shutdown_connections 2024-08-17T02:22:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f467c060860 0x7f467c062d10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4698158eb0 0x7f469815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46980aecc0 0x7f469815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 --2- 172.21.15.100:0/1318106587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f469806e620 0x7f4698140250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 >> 172.21.15.100:0/1318106587 conn(0x7f46980678d0 msgr2=0x7f4698068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 shutdown_connections 2024-08-17T02:22:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.168+0000 7f469f322700 1 -- 172.21.15.100:0/1318106587 wait complete. 2024-08-17T02:22:48.183 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-08-17T02:22:48.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d03fff700 1 Processor -- start 2024-08-17T02:22:48.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d03fff700 1 -- start start 2024-08-17T02:22:48.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d03fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d0406e620 0x7f1d040b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d03fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04141a00 con 0x7f1d040b0d20 2024-08-17T02:22:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d03fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04140b10 con 0x7f1d040b1630 2024-08-17T02:22:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d03fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04140df0 con 0x7f1d0406e620 2024-08-17T02:22:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d02ffd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1d040b0d20 0x7f1d0406df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48896/0 (socket says 172.21.15.100:48896) 2024-08-17T02:22:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d02ffd700 1 -- 172.21.15.100:0/2941090824 learned_addr learned my addr 172.21.15.100:0/2941090824 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d027fc700 1 --2- 172.21.15.100:0/2941090824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d0406e620 0x7f1d040b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d01ffb700 1 -- 172.21.15.100:0/2941090824 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354405725 0 0) 0x7f1d04141a00 con 0x7f1d040b0d20 2024-08-17T02:22:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d01ffb700 1 -- 172.21.15.100:0/2941090824 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cec003570 con 0x7f1d040b0d20 2024-08-17T02:22:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d01ffb700 1 -- 172.21.15.100:0/2941090824 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768503492 0 0) 0x7f1d04140b10 con 0x7f1d040b1630 2024-08-17T02:22:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d01ffb700 1 -- 172.21.15.100:0/2941090824 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d04141a00 con 0x7f1d040b1630 2024-08-17T02:22:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d027fc700 1 -- 172.21.15.100:0/2941090824 >> v1:172.21.15.123:6789/0 conn(0x7f1d040b1630 legacy=0x7f1d04141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d027fc700 1 -- 172.21.15.100:0/2941090824 >> v1:172.21.15.100:6789/0 conn(0x7f1d040b0d20 legacy=0x7f1d0406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.292+0000 7f1d027fc700 1 -- 172.21.15.100:0/2941090824 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d041410d0 con 0x7f1d0406e620 2024-08-17T02:22:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d027fc700 1 --2- 172.21.15.100:0/2941090824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d0406e620 0x7f1d040b10f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009910 tx=0x7f1cf40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ec5642b4774682 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d01ffb700 1 -- 172.21.15.100:0/2941090824 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4011630 con 0x7f1d0406e620 2024-08-17T02:22:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d01ffb700 1 -- 172.21.15.100:0/2941090824 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf4011790 con 0x7f1d0406e620 2024-08-17T02:22:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d01ffb700 1 -- 172.21.15.100:0/2941090824 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4011b10 con 0x7f1d0406e620 2024-08-17T02:22:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 -- 172.21.15.100:0/2941090824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d0406e620 msgr2=0x7f1d040b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 --2- 172.21.15.100:0/2941090824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d0406e620 0x7f1d040b10f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009910 tx=0x7f1cf40091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 -- 172.21.15.100:0/2941090824 shutdown_connections 2024-08-17T02:22:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 --2- 172.21.15.100:0/2941090824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d0406e620 0x7f1d040b10f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 -- 172.21.15.100:0/2941090824 >> 172.21.15.100:0/2941090824 conn(0x7f1d040678d0 msgr2=0x7f1d04067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 -- 172.21.15.100:0/2941090824 shutdown_connections 2024-08-17T02:22:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 -- 172.21.15.100:0/2941090824 wait complete. 2024-08-17T02:22:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 Processor -- start 2024-08-17T02:22:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.296+0000 7f1d03fff700 1 -- start start 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d0406e620 0x7f1d0414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d040b0d20 0x7f1d0406a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d040b1630 0x7f1d040ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040acd60 con 0x7f1d040b1630 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0414f810 con 0x7f1d0406e620 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040ac150 con 0x7f1d040b0d20 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d040b1630 0x7f1d040ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d040b1630 0x7f1d040ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38458/0 (socket says 172.21.15.100:38458) 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 -- 172.21.15.100:0/2924265462 learned_addr learned my addr 172.21.15.100:0/2924265462 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d027fc700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d040b0d20 0x7f1d0406a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d02ffd700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d0406e620 0x7f1d0414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 -- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d040b0d20 msgr2=0x7f1d0406a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d040b0d20 0x7f1d0406a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 -- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d0406e620 msgr2=0x7f1d0414f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d0406e620 0x7f1d0414f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 -- 172.21.15.100:0/2924265462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4009420 con 0x7f1d040b1630 2024-08-17T02:22:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d037fe700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d040b1630 0x7f1d040ab380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c910 tx=0x7f1cfc0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1ceaffd700 1 -- 172.21.15.100:0/2924265462 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc012630 con 0x7f1d040b1630 2024-08-17T02:22:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1ceaffd700 1 -- 172.21.15.100:0/2924265462 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc0127b0 con 0x7f1d040b1630 2024-08-17T02:22:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d040ac430 con 0x7f1d040b1630 2024-08-17T02:22:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1ceaffd700 1 -- 172.21.15.100:0/2924265462 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc012b70 con 0x7f1d040b1630 2024-08-17T02:22:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d040ac9f0 con 0x7f1d040b1630 2024-08-17T02:22:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d040aced0 con 0x7f1d040b1630 2024-08-17T02:22:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.300+0000 7f1ceaffd700 1 -- 172.21.15.100:0/2924265462 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1cfc01b400 con 0x7f1d040b1630 2024-08-17T02:22:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.304+0000 7f1ceaffd700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1cf0060860 0x7f1cf0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.304+0000 7f1ceaffd700 1 -- 172.21.15.100:0/2924265462 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1cfc07cc60 con 0x7f1d040b1630 2024-08-17T02:22:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.304+0000 7f1d02ffd700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1cf0060860 0x7f1cf0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.304+0000 7f1d02ffd700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1cf0060860 0x7f1cf0062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8007fd0 tx=0x7f1cf8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.308+0000 7f1ceaffd700 1 -- 172.21.15.100:0/2924265462 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1cfc04d3a0 con 0x7f1d040b1630 2024-08-17T02:22:48.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.528+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f1d040643a0 con 0x7f1d040b1630 2024-08-17T02:22:48.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.532+0000 7f1ceaffd700 1 -- 172.21.15.100:0/2924265462 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1cfc016020 con 0x7f1d040b1630 2024-08-17T02:22:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1cf0060860 msgr2=0x7f1cf0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1cf0060860 0x7f1cf0062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8007fd0 tx=0x7f1cf8009040 comp rx=0 tx=0).stop 2024-08-17T02:22:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d040b1630 msgr2=0x7f1d040ab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d040b1630 0x7f1d040ab380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c910 tx=0x7f1cfc0027a0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 shutdown_connections 2024-08-17T02:22:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1cf0060860 0x7f1cf0062d10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d040b1630 0x7f1d040ab380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d040b0d20 0x7f1d0406a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 --2- 172.21.15.100:0/2924265462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d0406e620 0x7f1d0414f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 >> 172.21.15.100:0/2924265462 conn(0x7f1d040678d0 msgr2=0x7f1d0415f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 shutdown_connections 2024-08-17T02:22:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.536+0000 7f1d03fff700 1 -- 172.21.15.100:0/2924265462 wait complete. 2024-08-17T02:22:48.550 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-08-17T02:22:48.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc5af83700 1 Processor -- start 2024-08-17T02:22:48.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc5af83700 1 -- start start 2024-08-17T02:22:48.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc5af83700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc540b0d20 0x7efc5406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc5af83700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 0x7efc540b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc5af83700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc54141a00 con 0x7efc540b1630 2024-08-17T02:22:48.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc5af83700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5406e8a0 con 0x7efc540b0d20 2024-08-17T02:22:48.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc5af83700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc54140b10 con 0x7efc5406e450 2024-08-17T02:22:48.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc59f81700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc540b0d20 0x7efc5406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc59f81700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc540b0d20 0x7efc5406df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38472/0 (socket says 172.21.15.100:38472) 2024-08-17T02:22:48.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc59f81700 1 -- 172.21.15.100:0/774901656 learned_addr learned my addr 172.21.15.100:0/774901656 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc59780700 1 --2- 172.21.15.100:0/774901656 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 0x7efc540b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc59f81700 1 -- 172.21.15.100:0/774901656 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 msgr2=0x7efc540b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc59f81700 1 --2- 172.21.15.100:0/774901656 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 0x7efc540b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc59f81700 1 -- 172.21.15.100:0/774901656 >> v1:172.21.15.123:6789/0 conn(0x7efc540b1630 legacy=0x7efc541610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.656+0000 7efc59f81700 1 -- 172.21.15.100:0/774901656 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc54140df0 con 0x7efc540b0d20 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc59f81700 1 --2- 172.21.15.100:0/774901656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc5406df10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009f20 tx=0x7efc4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc29198e6bf9237b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc58f7f700 1 -- 172.21.15.100:0/774901656 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00e070 con 0x7efc540b0d20 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc58f7f700 1 -- 172.21.15.100:0/774901656 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c00ba80 con 0x7efc540b0d20 2024-08-17T02:22:48.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc58f7f700 1 -- 172.21.15.100:0/774901656 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c0092e0 con 0x7efc540b0d20 2024-08-17T02:22:48.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 -- 172.21.15.100:0/774901656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 msgr2=0x7efc5406df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 --2- 172.21.15.100:0/774901656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc5406df10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009f20 tx=0x7efc4c009840 comp rx=0 tx=0).stop 2024-08-17T02:22:48.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 -- 172.21.15.100:0/774901656 shutdown_connections 2024-08-17T02:22:48.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 --2- 172.21.15.100:0/774901656 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 0x7efc540b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 --2- 172.21.15.100:0/774901656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc5406df10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 -- 172.21.15.100:0/774901656 >> 172.21.15.100:0/774901656 conn(0x7efc540678d0 msgr2=0x7efc54067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:48.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 -- 172.21.15.100:0/774901656 shutdown_connections 2024-08-17T02:22:48.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 -- 172.21.15.100:0/774901656 wait complete. 2024-08-17T02:22:48.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 Processor -- start 2024-08-17T02:22:48.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.660+0000 7efc5af83700 1 -- start start 2024-08-17T02:22:48.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 0x7efc540cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc54145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc540b1630 0x7efc541462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540cb5f0 con 0x7efc540b0d20 2024-08-17T02:22:48.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5415b450 con 0x7efc540b1630 2024-08-17T02:22:48.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5416f580 con 0x7efc5406e450 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc54145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc54145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38488/0 (socket says 172.21.15.100:38488) 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 -- 172.21.15.100:0/2018922741 learned_addr learned my addr 172.21.15.100:0/2018922741 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59f81700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 0x7efc540cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5a782700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc540b1630 0x7efc541462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 -- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 msgr2=0x7efc540cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 0x7efc540cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 -- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc540b1630 msgr2=0x7efc541462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc540b1630 0x7efc541462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 -- 172.21.15.100:0/2018922741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c009ae0 con 0x7efc540b0d20 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc59780700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc54145d70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc4400d8c0 tx=0x7efc44008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc49ffb700 1 -- 172.21.15.100:0/2018922741 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440125f0 con 0x7efc540b0d20 2024-08-17T02:22:48.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc49ffb700 1 -- 172.21.15.100:0/2018922741 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44012750 con 0x7efc540b0d20 2024-08-17T02:22:48.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc49ffb700 1 -- 172.21.15.100:0/2018922741 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44012b10 con 0x7efc540b0d20 2024-08-17T02:22:48.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc540a61f0 con 0x7efc540b0d20 2024-08-17T02:22:48.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5416ef30 con 0x7efc540b0d20 2024-08-17T02:22:48.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc540b1410 con 0x7efc540b0d20 2024-08-17T02:22:48.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.664+0000 7efc49ffb700 1 -- 172.21.15.100:0/2018922741 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efc4401e460 con 0x7efc540b0d20 2024-08-17T02:22:48.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.668+0000 7efc49ffb700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efc380608e0 0x7efc38062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:48.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.668+0000 7efc49ffb700 1 -- 172.21.15.100:0/2018922741 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7efc4407d850 con 0x7efc540b0d20 2024-08-17T02:22:48.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.668+0000 7efc59f81700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efc380608e0 0x7efc38062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:48.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.668+0000 7efc59f81700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efc380608e0 0x7efc38062d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009880 tx=0x7efc4c018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:48.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.672+0000 7efc49ffb700 1 -- 172.21.15.100:0/2018922741 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7efc44051a60 con 0x7efc540b0d20 2024-08-17T02:22:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.896+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7efc5406f5f0 con 0x7efc540b0d20 2024-08-17T02:22:48.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.900+0000 7efc49ffb700 1 -- 172.21.15.100:0/2018922741 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7efc440515f0 con 0x7efc540b0d20 2024-08-17T02:22:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efc380608e0 msgr2=0x7efc38062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efc380608e0 0x7efc38062d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009880 tx=0x7efc4c018000 comp rx=0 tx=0).stop 2024-08-17T02:22:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 msgr2=0x7efc54145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc54145d70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc4400d8c0 tx=0x7efc44008040 comp rx=0 tx=0).stop 2024-08-17T02:22:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 shutdown_connections 2024-08-17T02:22:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7efc380608e0 0x7efc38062d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc540b1630 0x7efc541462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc540b0d20 0x7efc54145d70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 --2- 172.21.15.100:0/2018922741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc5406e450 0x7efc540cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 >> 172.21.15.100:0/2018922741 conn(0x7efc540678d0 msgr2=0x7efc54141e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 shutdown_connections 2024-08-17T02:22:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:48.904+0000 7efc5af83700 1 -- 172.21.15.100:0/2018922741 wait complete. 2024-08-17T02:22:48.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-08-17T02:22:49.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5d0923700 1 Processor -- start 2024-08-17T02:22:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5d0923700 1 -- start start 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5d0923700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb5cc0b3c20 0x7fb5cc0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5d0923700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb5cc0b04e0 0x7fb5cc0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5d0923700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5cc0b0ea0 con 0x7fb5cc159340 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5d0923700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc0b45e0 con 0x7fb5cc0b3c20 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5d0923700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc0b1cf0 con 0x7fb5cc0b04e0 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5c9d9b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb5cc0b3c20 0x7fb5cc0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5ca59c700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb5cc159340 0x7fb5cc159710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48910/0 (socket says 172.21.15.100:48910) 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5ca59c700 1 -- 172.21.15.100:0/80771415 learned_addr learned my addr 172.21.15.100:0/80771415 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.023+0000 7fb5cad9d700 1 --2- 172.21.15.100:0/80771415 >> v2:172.21.15.123:3300/0 conn(0x7fb5cc0b04e0 0x7fb5cc0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5c9d9b700 1 -- 172.21.15.100:0/80771415 >> v2:172.21.15.123:3300/0 conn(0x7fb5cc0b04e0 msgr2=0x7fb5cc0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5c9d9b700 1 --2- 172.21.15.100:0/80771415 >> v2:172.21.15.123:3300/0 conn(0x7fb5cc0b04e0 0x7fb5cc0b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5c9d9b700 1 -- 172.21.15.100:0/80771415 >> v1:172.21.15.100:6789/0 conn(0x7fb5cc159340 legacy=0x7fb5cc159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5c9d9b700 1 -- 172.21.15.100:0/80771415 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc0b60d0 con 0x7fb5cc0b3c20 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5c9d9b700 1 --2- 172.21.15.100:0/80771415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc0b3c20 0x7fb5cc0b40a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc089490 tx=0x7fb5bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59500ee348c41a79 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5c959a700 1 -- 172.21.15.100:0/80771415 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc011630 con 0x7fb5cc0b3c20 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5c959a700 1 -- 172.21.15.100:0/80771415 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc011790 con 0x7fb5cc0b3c20 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5c959a700 1 -- 172.21.15.100:0/80771415 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc011b30 con 0x7fb5cc0b3c20 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5d0923700 1 -- 172.21.15.100:0/80771415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc0b3c20 msgr2=0x7fb5cc0b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5d0923700 1 --2- 172.21.15.100:0/80771415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc0b3c20 0x7fb5cc0b40a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc089490 tx=0x7fb5bc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5d0923700 1 -- 172.21.15.100:0/80771415 shutdown_connections 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5d0923700 1 --2- 172.21.15.100:0/80771415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc0b3c20 0x7fb5cc0b40a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5d0923700 1 --2- 172.21.15.100:0/80771415 >> v2:172.21.15.123:3300/0 conn(0x7fb5cc0b04e0 0x7fb5cc0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5d0923700 1 -- 172.21.15.100:0/80771415 >> 172.21.15.100:0/80771415 conn(0x7fb5cc0994f0 msgr2=0x7fb5cc0675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5d0923700 1 -- 172.21.15.100:0/80771415 shutdown_connections 2024-08-17T02:22:49.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.027+0000 7fb5d0923700 1 -- 172.21.15.100:0/80771415 wait complete. 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 Processor -- start 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 -- start start 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5cc0b04e0 0x7fb5cc16a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc0b3c20 0x7fb5cc169150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc159340 0x7fb5cc15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0c52f0 con 0x7fb5cc159340 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0ab740 con 0x7fb5cc0b3c20 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0aba20 con 0x7fb5cc0b04e0 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc159340 0x7fb5cc15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc159340 0x7fb5cc15a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38508/0 (socket says 172.21.15.100:38508) 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 -- 172.21.15.100:0/1120109885 learned_addr learned my addr 172.21.15.100:0/1120109885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5c9d9b700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc0b3c20 0x7fb5cc169150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5ca59c700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5cc0b04e0 0x7fb5cc16a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 -- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5cc0b04e0 msgr2=0x7fb5cc16a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5cc0b04e0 0x7fb5cc16a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 -- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc0b3c20 msgr2=0x7fb5cc169150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc0b3c20 0x7fb5cc169150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 -- 172.21.15.100:0/1120109885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc009420 con 0x7fb5cc159340 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5cad9d700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc159340 0x7fb5cc15a520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009e90 tx=0x7fb5c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5ba7fc700 1 -- 172.21.15.100:0/1120109885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c400cc20 con 0x7fb5cc159340 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5ba7fc700 1 -- 172.21.15.100:0/1120109885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c400f040 con 0x7fb5cc159340 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc0abd00 con 0x7fb5cc159340 2024-08-17T02:22:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5ba7fc700 1 -- 172.21.15.100:0/1120109885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4014480 con 0x7fb5cc159340 2024-08-17T02:22:49.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.031+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc16a9d0 con 0x7fb5cc159340 2024-08-17T02:22:49.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.035+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc0645c0 con 0x7fb5cc159340 2024-08-17T02:22:49.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.035+0000 7fb5ba7fc700 1 -- 172.21.15.100:0/1120109885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb5c4014830 con 0x7fb5cc159340 2024-08-17T02:22:49.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.035+0000 7fb5ba7fc700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb5ac060890 0x7fb5ac062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.035+0000 7fb5ba7fc700 1 -- 172.21.15.100:0/1120109885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb5c407d250 con 0x7fb5cc159340 2024-08-17T02:22:49.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.035+0000 7fb5ca59c700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb5ac060890 0x7fb5ac062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.035+0000 7fb5ca59c700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb5ac060890 0x7fb5ac062d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0001700 tx=0x7fb5c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.043+0000 7fb5ba7fc700 1 -- 172.21.15.100:0/1120109885 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb5c4051480 con 0x7fb5cc159340 2024-08-17T02:22:49.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.263+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fb5cc0d10f0 con 0x7fb5cc159340 2024-08-17T02:22:49.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.267+0000 7fb5ba7fc700 1 -- 172.21.15.100:0/1120109885 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb5c4051010 con 0x7fb5cc159340 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb5ac060890 msgr2=0x7fb5ac062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb5ac060890 0x7fb5ac062d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0001700 tx=0x7fb5c0009040 comp rx=0 tx=0).stop 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc159340 msgr2=0x7fb5cc15a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc159340 0x7fb5cc15a520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009e90 tx=0x7fb5c400b040 comp rx=0 tx=0).stop 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 shutdown_connections 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb5ac060890 0x7fb5ac062d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5cc159340 0x7fb5cc15a520 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc0b3c20 0x7fb5cc169150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 --2- 172.21.15.100:0/1120109885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5cc0b04e0 0x7fb5cc16a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 >> 172.21.15.100:0/1120109885 conn(0x7fb5cc0994f0 msgr2=0x7fb5cc159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:49.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 shutdown_connections 2024-08-17T02:22:49.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.271+0000 7fb5d0923700 1 -- 172.21.15.100:0/1120109885 wait complete. 2024-08-17T02:22:49.286 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:49.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842903700 1 Processor -- start 2024-08-17T02:22:49.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842903700 1 -- start start 2024-08-17T02:22:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842903700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f783c154900 0x7f783c0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842903700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f783c0cb560 0x7f783c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842903700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c0b1730 con 0x7f783c0b12d0 2024-08-17T02:22:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842903700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c068790 con 0x7f783c154900 2024-08-17T02:22:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842903700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c1583f0 con 0x7f783c0cb560 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7841100700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f783c0b12d0 0x7f783c0cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48920/0 (socket says 172.21.15.100:48920) 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7841100700 1 -- 172.21.15.100:0/1054683814 learned_addr learned my addr 172.21.15.100:0/1054683814 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7841901700 1 --2- 172.21.15.100:0/1054683814 >> v2:172.21.15.100:3300/0 conn(0x7f783c154900 0x7f783c0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f78408ff700 1 -- 172.21.15.100:0/1054683814 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377264225 0 0) 0x7f7838001110 con 0x7f783c0b12d0 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842102700 1 --2- 172.21.15.100:0/1054683814 >> v2:172.21.15.123:3300/0 conn(0x7f783c0cb560 0x7f783c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f78408ff700 1 -- 172.21.15.100:0/1054683814 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7824003570 con 0x7f783c0b12d0 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7841901700 1 -- 172.21.15.100:0/1054683814 >> v2:172.21.15.123:3300/0 conn(0x7f783c0cb560 msgr2=0x7f783c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f78408ff700 1 -- 172.21.15.100:0/1054683814 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1432049613 0 0) 0x7f7824003570 con 0x7f783c0b12d0 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7841901700 1 --2- 172.21.15.100:0/1054683814 >> v2:172.21.15.123:3300/0 conn(0x7f783c0cb560 0x7f783c160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7841901700 1 -- 172.21.15.100:0/1054683814 >> v1:172.21.15.100:6789/0 conn(0x7f783c0b12d0 legacy=0x7f783c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7841901700 1 -- 172.21.15.100:0/1054683814 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c0cbaa0 con 0x7f783c154900 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7842102700 1 --2- 172.21.15.100:0/1054683814 >> v2:172.21.15.123:3300/0 conn(0x7f783c0cb560 0x7f783c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f7841901700 1 --2- 172.21.15.100:0/1054683814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c154900 0x7f783c0b0d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f783c06c980 tx=0x7f782c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89328f3e87ecbbd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f78408ff700 1 -- 172.21.15.100:0/1054683814 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838001600 con 0x7f783c154900 2024-08-17T02:22:49.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f78408ff700 1 -- 172.21.15.100:0/1054683814 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c011420 con 0x7f783c154900 2024-08-17T02:22:49.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.387+0000 7f78408ff700 1 -- 172.21.15.100:0/1054683814 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c0117c0 con 0x7f783c154900 2024-08-17T02:22:49.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- 172.21.15.100:0/1054683814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c154900 msgr2=0x7f783c0b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 --2- 172.21.15.100:0/1054683814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c154900 0x7f783c0b0d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f783c06c980 tx=0x7f782c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- 172.21.15.100:0/1054683814 shutdown_connections 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 --2- 172.21.15.100:0/1054683814 >> v2:172.21.15.123:3300/0 conn(0x7f783c0cb560 0x7f783c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 --2- 172.21.15.100:0/1054683814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c154900 0x7f783c0b0d90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- 172.21.15.100:0/1054683814 >> 172.21.15.100:0/1054683814 conn(0x7f783c0678d0 msgr2=0x7f783c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- 172.21.15.100:0/1054683814 shutdown_connections 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- 172.21.15.100:0/1054683814 wait complete. 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 Processor -- start 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- start start 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c0b12d0 0x7f783c1480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f783c0cb560 0x7f783c168ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f783c154900 0x7f783c0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0aeb80 con 0x7f783c0b12d0 2024-08-17T02:22:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0aee60 con 0x7f783c0cb560 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c169530 con 0x7f783c154900 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c0b12d0 0x7f783c1480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c0b12d0 0x7f783c1480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38528/0 (socket says 172.21.15.100:38528) 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 -- 172.21.15.100:0/1570898677 learned_addr learned my addr 172.21.15.100:0/1570898677 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 -- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f783c154900 msgr2=0x7f783c0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f783c154900 0x7f783c0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 -- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f783c0cb560 msgr2=0x7f783c168ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f783c0cb560 0x7f783c168ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 -- 172.21.15.100:0/1570898677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c009420 con 0x7f783c0b12d0 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7841901700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c0b12d0 0x7f783c1480a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f782c0091e0 tx=0x7f782c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7831ffb700 1 -- 172.21.15.100:0/1570898677 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c01cc50 con 0x7f783c0b12d0 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7831ffb700 1 -- 172.21.15.100:0/1570898677 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c01cdb0 con 0x7f783c0b12d0 2024-08-17T02:22:49.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7831ffb700 1 -- 172.21.15.100:0/1570898677 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c01a6a0 con 0x7f783c0b12d0 2024-08-17T02:22:49.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c0aa310 con 0x7f783c0b12d0 2024-08-17T02:22:49.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.391+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c0aa8d0 con 0x7f783c0b12d0 2024-08-17T02:22:49.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.395+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c0643a0 con 0x7f783c0b12d0 2024-08-17T02:22:49.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.395+0000 7f7831ffb700 1 -- 172.21.15.100:0/1570898677 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f782c01aad0 con 0x7f783c0b12d0 2024-08-17T02:22:49.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.395+0000 7f7831ffb700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7828060860 0x7f7828062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.395+0000 7f7831ffb700 1 -- 172.21.15.100:0/1570898677 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f782c07e720 con 0x7f783c0b12d0 2024-08-17T02:22:49.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.395+0000 7f7841100700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7828060860 0x7f7828062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.395+0000 7f7841100700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7828060860 0x7f7828062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7838001ed0 tx=0x7f7838000fe0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.403+0000 7f7831ffb700 1 -- 172.21.15.100:0/1570898677 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f782c04ee20 con 0x7f783c0b12d0 2024-08-17T02:22:49.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.567+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f783c06f5f0 con 0x7f783c0b12d0 2024-08-17T02:22:49.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.567+0000 7f7831ffb700 1 -- 172.21.15.100:0/1570898677 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f782c052440 con 0x7f783c0b12d0 2024-08-17T02:22:49.572 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:49.572 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":29,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:46.640891+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7828060860 msgr2=0x7f7828062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7828060860 0x7f7828062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7838001ed0 tx=0x7f7838000fe0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c0b12d0 msgr2=0x7f783c1480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c0b12d0 0x7f783c1480a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f782c0091e0 tx=0x7f782c016000 comp rx=0 tx=0).stop 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 shutdown_connections 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7828060860 0x7f7828062d10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f783c154900 0x7f783c0b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f783c0cb560 0x7f783c168ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 --2- 172.21.15.100:0/1570898677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f783c0b12d0 0x7f783c1480a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 >> 172.21.15.100:0/1570898677 conn(0x7f783c0678d0 msgr2=0x7f783c0aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 shutdown_connections 2024-08-17T02:22:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.571+0000 7f7842903700 1 -- 172.21.15.100:0/1570898677 wait complete. 2024-08-17T02:22:49.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-17T02:22:49.587 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:22:49.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f970cb07700 1 Processor -- start 2024-08-17T02:22:49.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f970cb07700 1 -- start start 2024-08-17T02:22:49.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f970cb07700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97081590a0 0x7f9708159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f970cb07700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080cf520 con 0x7f9708066e00 2024-08-17T02:22:49.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f970cb07700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080d0f40 con 0x7f97080cf9b0 2024-08-17T02:22:49.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f970cb07700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97080ce690 con 0x7f97081590a0 2024-08-17T02:22:49.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97077fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97081590a0 0x7f9708159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f9707fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9708066e00 0x7f97080afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48928/0 (socket says 172.21.15.100:48928) 2024-08-17T02:22:49.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f9707fff700 1 -- 172.21.15.100:0/716047888 learned_addr learned my addr 172.21.15.100:0/716047888 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:49.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97077fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97081590a0 0x7f9708159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38540/0 (socket says 172.21.15.100:38540) 2024-08-17T02:22:49.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97067fc700 1 -- 172.21.15.100:0/716047888 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164404666 0 0) 0x7f97080cf520 con 0x7f9708066e00 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97067fc700 1 -- 172.21.15.100:0/716047888 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f4003570 con 0x7f9708066e00 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97077fe700 1 -- 172.21.15.100:0/716047888 >> v1:172.21.15.123:6789/0 conn(0x7f97080cf9b0 legacy=0x7f97080cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97077fe700 1 -- 172.21.15.100:0/716047888 >> v1:172.21.15.100:6789/0 conn(0x7f9708066e00 legacy=0x7f97080afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97077fe700 1 -- 172.21.15.100:0/716047888 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080ce910 con 0x7f97081590a0 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97077fe700 1 --2- 172.21.15.100:0/716047888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97081590a0 0x7f9708159470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96f80098e0 tx=0x7f96f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23448421416bbc03 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.695+0000 7f97067fc700 1 -- 172.21.15.100:0/716047888 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800b640 con 0x7f97081590a0 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f97067fc700 1 -- 172.21.15.100:0/716047888 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f800b7a0 con 0x7f97081590a0 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f97067fc700 1 -- 172.21.15.100:0/716047888 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800bb20 con 0x7f97081590a0 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 -- 172.21.15.100:0/716047888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97081590a0 msgr2=0x7f9708159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 --2- 172.21.15.100:0/716047888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97081590a0 0x7f9708159470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96f80098e0 tx=0x7f96f80091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 -- 172.21.15.100:0/716047888 shutdown_connections 2024-08-17T02:22:49.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 --2- 172.21.15.100:0/716047888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97081590a0 0x7f9708159470 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 -- 172.21.15.100:0/716047888 >> 172.21.15.100:0/716047888 conn(0x7f9708068c80 msgr2=0x7f97080d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:49.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 -- 172.21.15.100:0/716047888 shutdown_connections 2024-08-17T02:22:49.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 -- 172.21.15.100:0/716047888 wait complete. 2024-08-17T02:22:49.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 Processor -- start 2024-08-17T02:22:49.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 -- start start 2024-08-17T02:22:49.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.699+0000 7f970cb07700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708066e00 0x7f97080b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f970cb07700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97080cf9b0 0x7f970816ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f970cb07700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97081590a0 0x7f970816b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f970cb07700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080b0ea0 con 0x7f97080cf9b0 2024-08-17T02:22:49.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f970cb07700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080b0290 con 0x7f9708066e00 2024-08-17T02:22:49.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f970cb07700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97080b0570 con 0x7f97081590a0 2024-08-17T02:22:49.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97080cf9b0 0x7f970816ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97080cf9b0 0x7f970816ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38546/0 (socket says 172.21.15.100:38546) 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 -- 172.21.15.100:0/3308925617 learned_addr learned my addr 172.21.15.100:0/3308925617 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f97077fe700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708066e00 0x7f97080b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9707fff700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97081590a0 0x7f970816b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 -- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97081590a0 msgr2=0x7f970816b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97081590a0 0x7f970816b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 -- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708066e00 msgr2=0x7f97080b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708066e00 0x7f97080b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 -- 172.21.15.100:0/3308925617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8009420 con 0x7f97080cf9b0 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f9706ffd700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97080cf9b0 0x7f970816ac40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96f000c590 tx=0x7f96f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f96ff7fe700 1 -- 172.21.15.100:0/3308925617 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0011590 con 0x7f97080cf9b0 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97080b0850 con 0x7f97080cf9b0 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f96ff7fe700 1 -- 172.21.15.100:0/3308925617 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0011710 con 0x7f97080cf9b0 2024-08-17T02:22:49.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f96ff7fe700 1 -- 172.21.15.100:0/3308925617 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0011ad0 con 0x7f97080cf9b0 2024-08-17T02:22:49.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970814d080 con 0x7f97080cf9b0 2024-08-17T02:22:49.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97080b1050 con 0x7f97080cf9b0 2024-08-17T02:22:49.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.703+0000 7f96ff7fe700 1 -- 172.21.15.100:0/3308925617 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f96f001b3b0 con 0x7f97080cf9b0 2024-08-17T02:22:49.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.707+0000 7f96ff7fe700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f96e8060860 0x7f96e8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:49.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.707+0000 7f96ff7fe700 1 -- 172.21.15.100:0/3308925617 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f96f007be00 con 0x7f97080cf9b0 2024-08-17T02:22:49.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.707+0000 7f97077fe700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f96e8060860 0x7f96e8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:49.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.707+0000 7f97077fe700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f96e8060860 0x7f96e8062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96f80093f0 tx=0x7f96f8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:49.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.715+0000 7f96ff7fe700 1 -- 172.21.15.100:0/3308925617 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f97080b1050 con 0x7f97080cf9b0 2024-08-17T02:22:49.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.879+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97080b1050 con 0x7f97080cf9b0 2024-08-17T02:22:49.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.879+0000 7f96ff7fe700 1 -- 172.21.15.100:0/3308925617 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97080b1050 con 0x7f97080cf9b0 2024-08-17T02:22:49.882 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.879+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f96e8060860 msgr2=0x7f96e8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.879+0000 7f970cb07700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f96e8060860 0x7f96e8062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96f80093f0 tx=0x7f96f8017000 comp rx=0 tx=0).stop 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.879+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97080cf9b0 msgr2=0x7f970816ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.879+0000 7f970cb07700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97080cf9b0 0x7f970816ac40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96f000c590 tx=0x7f96f000c8a0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.879+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 shutdown_connections 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.879+0000 7f970cb07700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f96e8060860 0x7f96e8062d10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.883+0000 7f970cb07700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97081590a0 0x7f970816b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.883+0000 7f970cb07700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97080cf9b0 0x7f970816ac40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.883+0000 7f970cb07700 1 --2- 172.21.15.100:0/3308925617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708066e00 0x7f97080b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:49.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.883+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 >> 172.21.15.100:0/3308925617 conn(0x7f9708068c80 msgr2=0x7f9708154310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:49.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.883+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 shutdown_connections 2024-08-17T02:22:49.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:49.883+0000 7f970cb07700 1 -- 172.21.15.100:0/3308925617 wait complete. 2024-08-17T02:22:49.897 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-17T02:22:50.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b8028700 1 Processor -- start 2024-08-17T02:22:50.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b8028700 1 -- start start 2024-08-17T02:22:50.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b8028700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b00aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b8028700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40b015a5b0 0x7f40b00b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b8028700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00aef60 con 0x7f40b00b3360 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b8028700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b0068f10 con 0x7f40b015a5b0 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b8028700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00b3800 con 0x7f40b00cd760 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b55c3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40b015a5b0 0x7f40b00b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b55c3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40b015a5b0 0x7f40b00b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38552/0 (socket says 172.21.15.100:38552) 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b55c3700 1 -- 172.21.15.100:0/3339545029 learned_addr learned my addr 172.21.15.100:0/3339545029 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b5dc4700 1 --2- 172.21.15.100:0/3339545029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b00aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b55c3700 1 -- 172.21.15.100:0/3339545029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 msgr2=0x7f40b00aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b55c3700 1 --2- 172.21.15.100:0/3339545029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b00aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b55c3700 1 -- 172.21.15.100:0/3339545029 >> v1:172.21.15.123:6789/0 conn(0x7f40b00b3360 legacy=0x7f40b00b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.007+0000 7f40b55c3700 1 -- 172.21.15.100:0/3339545029 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00b3ae0 con 0x7f40b015a5b0 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b55c3700 1 --2- 172.21.15.100:0/3339545029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b015a5b0 0x7f40b00b2e20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f40a40099b0 tx=0x7f40a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25555311742ca127 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b4dc2700 1 -- 172.21.15.100:0/3339545029 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4009ea0 con 0x7f40b015a5b0 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b4dc2700 1 -- 172.21.15.100:0/3339545029 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a400b860 con 0x7f40b015a5b0 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b4dc2700 1 -- 172.21.15.100:0/3339545029 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a400bc00 con 0x7f40b015a5b0 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 -- 172.21.15.100:0/3339545029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b015a5b0 msgr2=0x7f40b00b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 --2- 172.21.15.100:0/3339545029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b015a5b0 0x7f40b00b2e20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f40a40099b0 tx=0x7f40a4009250 comp rx=0 tx=0).stop 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 -- 172.21.15.100:0/3339545029 shutdown_connections 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 --2- 172.21.15.100:0/3339545029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b015a5b0 0x7f40b00b2e20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 --2- 172.21.15.100:0/3339545029 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b00aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 -- 172.21.15.100:0/3339545029 >> 172.21.15.100:0/3339545029 conn(0x7f40b00ce050 msgr2=0x7f40b00ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 -- 172.21.15.100:0/3339545029 shutdown_connections 2024-08-17T02:22:50.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 -- 172.21.15.100:0/3339545029 wait complete. 2024-08-17T02:22:50.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.011+0000 7f40b8028700 1 Processor -- start 2024-08-17T02:22:50.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 -- start start 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b00b3360 0x7f40b015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b015a5b0 0x7f40b0152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b01509d0 con 0x7f40b00b3360 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b015c4c0 con 0x7f40b015a5b0 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b015c780 con 0x7f40b00cd760 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b5dc4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b00b3360 0x7f40b015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b5dc4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b00b3360 0x7f40b015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38558/0 (socket says 172.21.15.100:38558) 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b5dc4700 1 -- 172.21.15.100:0/1411781925 learned_addr learned my addr 172.21.15.100:0/1411781925 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b55c3700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b65c5700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b015a5b0 0x7f40b0152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b55c3700 1 -- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b015a5b0 msgr2=0x7f40b0152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b55c3700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b015a5b0 0x7f40b0152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b55c3700 1 -- 172.21.15.100:0/1411781925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b00b3360 msgr2=0x7f40b015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b55c3700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b00b3360 0x7f40b015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b55c3700 1 -- 172.21.15.100:0/1411781925 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40094f0 con 0x7f40b00cd760 2024-08-17T02:22:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b5dc4700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b00b3360 0x7f40b015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b55c3700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b014e0b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40b0159970 tx=0x7f40a4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40adffb700 1 -- 172.21.15.100:0/1411781925 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a401be00 con 0x7f40b00cd760 2024-08-17T02:22:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40adffb700 1 -- 172.21.15.100:0/1411781925 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a400b9d0 con 0x7f40b00cd760 2024-08-17T02:22:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b015ca60 con 0x7f40b00cd760 2024-08-17T02:22:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40adffb700 1 -- 172.21.15.100:0/1411781925 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4026430 con 0x7f40b00cd760 2024-08-17T02:22:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.015+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b0154ff0 con 0x7f40b00cd760 2024-08-17T02:22:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.019+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b00b1380 con 0x7f40b00cd760 2024-08-17T02:22:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.019+0000 7f40adffb700 1 -- 172.21.15.100:0/1411781925 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f40a4023030 con 0x7f40b00cd760 2024-08-17T02:22:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.019+0000 7f40adffb700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f409c0608e0 0x7f409c062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.019+0000 7f40adffb700 1 -- 172.21.15.100:0/1411781925 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f40a407dfa0 con 0x7f40b00cd760 2024-08-17T02:22:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.019+0000 7f40b5dc4700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f409c0608e0 0x7f409c062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.019+0000 7f40b5dc4700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f409c0608e0 0x7f409c062d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009fd0 tx=0x7f40a000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:50.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.027+0000 7f40adffb700 1 -- 172.21.15.100:0/1411781925 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f40a404e7a0 con 0x7f40b00cd760 2024-08-17T02:22:50.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.191+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f40b00b1380 con 0x7f40b00cd760 2024-08-17T02:22:50.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.191+0000 7f40adffb700 1 -- 172.21.15.100:0/1411781925 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f40b00b1380 con 0x7f40b00cd760 2024-08-17T02:22:50.197 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-08-17T02:22:50.197 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:22:50.197 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-08-17T02:22:50.197 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-08-17T02:22:50.197 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-08-17T02:22:50.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f409c0608e0 msgr2=0x7f409c062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f409c0608e0 0x7f409c062d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009fd0 tx=0x7f40a000b040 comp rx=0 tx=0).stop 2024-08-17T02:22:50.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 msgr2=0x7f40b014e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b014e0b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40b0159970 tx=0x7f40a4017000 comp rx=0 tx=0).stop 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 shutdown_connections 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f409c0608e0 0x7f409c062d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b015a5b0 0x7f40b0152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40b00cd760 0x7f40b014e0b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 --2- 172.21.15.100:0/1411781925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b00b3360 0x7f40b015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 >> 172.21.15.100:0/1411781925 conn(0x7f40b00ce050 msgr2=0x7f40b014ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 shutdown_connections 2024-08-17T02:22:50.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.195+0000 7f40b8028700 1 -- 172.21.15.100:0/1411781925 wait complete. 2024-08-17T02:22:50.210 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-17T02:22:50.211 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-17T02:22:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27621b4700 1 Processor -- start 2024-08-17T02:22:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27621b4700 1 -- start start 2024-08-17T02:22:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27621b4700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f275c155410 0x7f275c0aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27621b4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c068db0 0x7f275c06a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27621b4700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275c069d00 con 0x7f275c1557e0 2024-08-17T02:22:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27621b4700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275c0699d0 con 0x7f275c155410 2024-08-17T02:22:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27621b4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0aba10 con 0x7f275c068db0 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27611b2700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f275c155410 0x7f275c0aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27611b2700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f275c155410 0x7f275c0aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38568/0 (socket says 172.21.15.100:38568) 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.319+0000 7f27611b2700 1 -- 172.21.15.100:0/3423518550 learned_addr learned my addr 172.21.15.100:0/3423518550 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27619b3700 1 --2- 172.21.15.100:0/3423518550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c068db0 0x7f275c06a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27611b2700 1 -- 172.21.15.100:0/3423518550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c068db0 msgr2=0x7f275c06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27611b2700 1 --2- 172.21.15.100:0/3423518550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c068db0 0x7f275c06a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27611b2700 1 -- 172.21.15.100:0/3423518550 >> v1:172.21.15.100:6789/0 conn(0x7f275c1557e0 legacy=0x7f275c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27611b2700 1 -- 172.21.15.100:0/3423518550 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0acf90 con 0x7f275c155410 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27619b3700 1 --2- 172.21.15.100:0/3423518550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c068db0 0x7f275c06a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27611b2700 1 --2- 172.21.15.100:0/3423518550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c155410 0x7f275c0aca50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f275800ab50 tx=0x7f275800a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8751fbd09190d21e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:50.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f2753fff700 1 -- 172.21.15.100:0/3423518550 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275800ca60 con 0x7f275c155410 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f2753fff700 1 -- 172.21.15.100:0/3423518550 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275800cbc0 con 0x7f275c155410 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f2753fff700 1 -- 172.21.15.100:0/3423518550 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758018400 con 0x7f275c155410 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27621b4700 1 -- 172.21.15.100:0/3423518550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c155410 msgr2=0x7f275c0aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27621b4700 1 --2- 172.21.15.100:0/3423518550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c155410 0x7f275c0aca50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f275800ab50 tx=0x7f275800a3f0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27621b4700 1 -- 172.21.15.100:0/3423518550 shutdown_connections 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27621b4700 1 --2- 172.21.15.100:0/3423518550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c068db0 0x7f275c06a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27621b4700 1 --2- 172.21.15.100:0/3423518550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c155410 0x7f275c0aca50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27621b4700 1 -- 172.21.15.100:0/3423518550 >> 172.21.15.100:0/3423518550 conn(0x7f275c0ccfa0 msgr2=0x7f275c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27621b4700 1 -- 172.21.15.100:0/3423518550 shutdown_connections 2024-08-17T02:22:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.323+0000 7f27621b4700 1 -- 172.21.15.100:0/3423518550 wait complete. 2024-08-17T02:22:50.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 Processor -- start 2024-08-17T02:22:50.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 -- start start 2024-08-17T02:22:50.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c068db0 0x7f275c0aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c155410 0x7f275c152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f275c1557e0 0x7f275c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c153810 con 0x7f275c068db0 2024-08-17T02:22:50.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c151f80 con 0x7f275c1557e0 2024-08-17T02:22:50.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0ab430 con 0x7f275c155410 2024-08-17T02:22:50.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c068db0 0x7f275c0aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c068db0 0x7f275c0aaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38572/0 (socket says 172.21.15.100:38572) 2024-08-17T02:22:50.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 -- 172.21.15.100:0/1983711860 learned_addr learned my addr 172.21.15.100:0/1983711860 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:50.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27619b3700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f275c1557e0 0x7f275c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27609b1700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c155410 0x7f275c152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 -- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c155410 msgr2=0x7f275c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c155410 0x7f275c152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 -- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f275c1557e0 msgr2=0x7f275c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f275c1557e0 0x7f275c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 -- 172.21.15.100:0/1983711860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275800a690 con 0x7f275c068db0 2024-08-17T02:22:50.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27609b1700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c155410 0x7f275c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:50.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27619b3700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f275c1557e0 0x7f275c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:50.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27611b2700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c068db0 0x7f275c0aaef0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f275c0694d0 tx=0x7f275801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:50.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27517fa700 1 -- 172.21.15.100:0/1983711860 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275801c040 con 0x7f275c068db0 2024-08-17T02:22:50.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27517fa700 1 -- 172.21.15.100:0/1983711860 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27580106d0 con 0x7f275c068db0 2024-08-17T02:22:50.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.327+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c151650 con 0x7f275c068db0 2024-08-17T02:22:50.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.331+0000 7f27517fa700 1 -- 172.21.15.100:0/1983711860 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758010a70 con 0x7f275c068db0 2024-08-17T02:22:50.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.331+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c151c10 con 0x7f275c068db0 2024-08-17T02:22:50.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.331+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c0ad570 con 0x7f275c068db0 2024-08-17T02:22:50.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.331+0000 7f27517fa700 1 -- 172.21.15.100:0/1983711860 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2758021480 con 0x7f275c068db0 2024-08-17T02:22:50.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.331+0000 7f27517fa700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2740060860 0x7f2740062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:50.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.331+0000 7f27517fa700 1 -- 172.21.15.100:0/1983711860 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f275807e180 con 0x7f275c068db0 2024-08-17T02:22:50.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.331+0000 7f27609b1700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2740060860 0x7f2740062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:50.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.335+0000 7f27609b1700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2740060860 0x7f2740062d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f275c06c160 tx=0x7f274c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:50.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.339+0000 7f27517fa700 1 -- 172.21.15.100:0/1983711860 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f275c0ad570 con 0x7f275c068db0 2024-08-17T02:22:50.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.499+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f275c0ad570 con 0x7f275c068db0 2024-08-17T02:22:50.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.907+0000 7f27517fa700 1 -- 172.21.15.100:0/1983711860 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f275c0ad570 con 0x7f275c068db0 2024-08-17T02:22:50.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2740060860 msgr2=0x7f2740062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2740060860 0x7f2740062d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f275c06c160 tx=0x7f274c009040 comp rx=0 tx=0).stop 2024-08-17T02:22:50.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c068db0 msgr2=0x7f275c0aaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:50.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c068db0 0x7f275c0aaef0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f275c0694d0 tx=0x7f275801a000 comp rx=0 tx=0).stop 2024-08-17T02:22:50.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 shutdown_connections 2024-08-17T02:22:50.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2740060860 0x7f2740062d10 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f275c06c160 tx=0x7f274c009040 comp rx=0 tx=0).stop 2024-08-17T02:22:50.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f275c1557e0 0x7f275c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f275c155410 0x7f275c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 --2- 172.21.15.100:0/1983711860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275c068db0 0x7f275c0aaef0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:50.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.915+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 >> 172.21.15.100:0/1983711860 conn(0x7f275c0ccfa0 msgr2=0x7f275c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:50.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.919+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 shutdown_connections 2024-08-17T02:22:50.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:50.919+0000 7f27621b4700 1 -- 172.21.15.100:0/1983711860 wait complete. 2024-08-17T02:22:50.922 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-08-17T02:22:50.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-17T02:22:51.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.047+0000 7f6b7759e700 1 Processor -- start 2024-08-17T02:22:51.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.047+0000 7f6b7759e700 1 -- start start 2024-08-17T02:22:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6b781534b0 0x7f6b780cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b780b21d0 0x7f6b780d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780ce7f0 con 0x7f6b78153880 2024-08-17T02:22:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b780cea90 con 0x7f6b781534b0 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780ced70 con 0x7f6b780b21d0 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b76d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b780b21d0 0x7f6b780d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b76d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b780b21d0 0x7f6b780d0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:45970/0 (socket says 172.21.15.100:45970) 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b76d9d700 1 -- 172.21.15.100:0/1119868012 learned_addr learned my addr 172.21.15.100:0/1119868012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6b781534b0 0x7f6b780cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7559a700 1 -- 172.21.15.100:0/1119868012 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014000310 0 0) 0x7f6b780ce7f0 con 0x7f6b78153880 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b76d9d700 1 -- 172.21.15.100:0/1119868012 >> v2:172.21.15.123:3300/0 conn(0x7f6b781534b0 msgr2=0x7f6b780cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b76d9d700 1 --2- 172.21.15.100:0/1119868012 >> v2:172.21.15.123:3300/0 conn(0x7f6b781534b0 0x7f6b780cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b76d9d700 1 -- 172.21.15.100:0/1119868012 >> v1:172.21.15.123:6789/0 conn(0x7f6b78153880 legacy=0x7f6b780b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b76d9d700 1 -- 172.21.15.100:0/1119868012 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780cf050 con 0x7f6b780b21d0 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b76d9d700 1 --2- 172.21.15.100:0/1119868012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b780b21d0 0x7f6b780d0130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009f20 tx=0x7f6b70009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d59e4677bde62c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7559a700 1 -- 172.21.15.100:0/1119868012 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7000e070 con 0x7f6b780b21d0 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7559a700 1 -- 172.21.15.100:0/1119868012 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7000b9d0 con 0x7f6b780b21d0 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7559a700 1 -- 172.21.15.100:0/1119868012 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b700092e0 con 0x7f6b780b21d0 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 -- 172.21.15.100:0/1119868012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b780b21d0 msgr2=0x7f6b780d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 --2- 172.21.15.100:0/1119868012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b780b21d0 0x7f6b780d0130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009f20 tx=0x7f6b70009840 comp rx=0 tx=0).stop 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 -- 172.21.15.100:0/1119868012 shutdown_connections 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 --2- 172.21.15.100:0/1119868012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b780b21d0 0x7f6b780d0130 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 --2- 172.21.15.100:0/1119868012 >> v2:172.21.15.123:3300/0 conn(0x7f6b781534b0 0x7f6b780cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 -- 172.21.15.100:0/1119868012 >> 172.21.15.100:0/1119868012 conn(0x7f6b78066e00 msgr2=0x7f6b78067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 -- 172.21.15.100:0/1119868012 shutdown_connections 2024-08-17T02:22:51.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.051+0000 7f6b7759e700 1 -- 172.21.15.100:0/1119868012 wait complete. 2024-08-17T02:22:51.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7759e700 1 Processor -- start 2024-08-17T02:22:51.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7759e700 1 -- start start 2024-08-17T02:22:51.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b780b21d0 0x7f6b78147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b781534b0 0x7f6b78148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b78153880 0x7f6b78144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7759e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78148880 con 0x7f6b780b21d0 2024-08-17T02:22:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78149680 con 0x7f6b781534b0 2024-08-17T02:22:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78148a70 con 0x7f6b78153880 2024-08-17T02:22:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b780b21d0 0x7f6b78147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b75d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b781534b0 0x7f6b78148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b76d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b78153880 0x7f6b78144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:51.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b780b21d0 0x7f6b78147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38574/0 (socket says 172.21.15.100:38574) 2024-08-17T02:22:51.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7659c700 1 -- 172.21.15.100:0/3102462824 learned_addr learned my addr 172.21.15.100:0/3102462824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:51.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7659c700 1 -- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b78153880 msgr2=0x7f6b78144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:51.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7659c700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b78153880 0x7f6b78144e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7659c700 1 -- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b781534b0 msgr2=0x7f6b78148340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:51.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7659c700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b781534b0 0x7f6b78148340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b7659c700 1 -- 172.21.15.100:0/3102462824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b70009ae0 con 0x7f6b780b21d0 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.055+0000 7f6b75d9b700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b781534b0 0x7f6b78148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.059+0000 7f6b7659c700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b780b21d0 0x7f6b78147a50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00ec80 tx=0x7f6b6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.059+0000 7f6b667fc700 1 -- 172.21.15.100:0/3102462824 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00c850 con 0x7f6b780b21d0 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.059+0000 7f6b667fc700 1 -- 172.21.15.100:0/3102462824 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c00c9b0 con 0x7f6b780b21d0 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.059+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b78148d50 con 0x7f6b780b21d0 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.059+0000 7f6b667fc700 1 -- 172.21.15.100:0/3102462824 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00cd50 con 0x7f6b780b21d0 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.059+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b78149310 con 0x7f6b780b21d0 2024-08-17T02:22:51.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.059+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b780643d0 con 0x7f6b780b21d0 2024-08-17T02:22:51.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.063+0000 7f6b667fc700 1 -- 172.21.15.100:0/3102462824 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6b6c013680 con 0x7f6b780b21d0 2024-08-17T02:22:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.063+0000 7f6b667fc700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6b58060860 0x7f6b58062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.063+0000 7f6b667fc700 1 -- 172.21.15.100:0/3102462824 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f6b6c07d390 con 0x7f6b780b21d0 2024-08-17T02:22:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.063+0000 7f6b75d9b700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6b58060860 0x7f6b58062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:51.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.063+0000 7f6b75d9b700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6b58060860 0x7f6b58062d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b7815f350 tx=0x7f6b68009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:51.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.067+0000 7f6b667fc700 1 -- 172.21.15.100:0/3102462824 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6b6c04da40 con 0x7f6b780b21d0 2024-08-17T02:22:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.231+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6b7814d290 con 0x7f6b780b21d0 2024-08-17T02:22:51.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.923+0000 7f6b667fc700 1 -- 172.21.15.100:0/3102462824 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6b6c051060 con 0x7f6b780b21d0 2024-08-17T02:22:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.931+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6b58060860 msgr2=0x7f6b58062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.931+0000 7f6b7759e700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6b58060860 0x7f6b58062d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b7815f350 tx=0x7f6b68009040 comp rx=0 tx=0).stop 2024-08-17T02:22:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.931+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b780b21d0 msgr2=0x7f6b78147a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.931+0000 7f6b7759e700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b780b21d0 0x7f6b78147a50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00ec80 tx=0x7f6b6c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.935+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 shutdown_connections 2024-08-17T02:22:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.935+0000 7f6b7759e700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6b58060860 0x7f6b58062d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.935+0000 7f6b7759e700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b78153880 0x7f6b78144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.935+0000 7f6b7759e700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b781534b0 0x7f6b78148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.935+0000 7f6b7759e700 1 --2- 172.21.15.100:0/3102462824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b780b21d0 0x7f6b78147a50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.935+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 >> 172.21.15.100:0/3102462824 conn(0x7f6b78066e00 msgr2=0x7f6b7815f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:51.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.935+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 shutdown_connections 2024-08-17T02:22:51.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:51.939+0000 7f6b7759e700 1 -- 172.21.15.100:0/3102462824 wait complete. 2024-08-17T02:22:51.943 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-08-17T02:22:51.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-17T02:22:52.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.063+0000 7fbf216cc700 1 Processor -- start 2024-08-17T02:22:52.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.063+0000 7fbf216cc700 1 -- start start 2024-08-17T02:22:52.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbf1c159fb0 0x7fbf1c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:52.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 0x7fbf1c155c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:52.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0ac7f0 con 0x7fbf1c159be0 2024-08-17T02:22:52.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1c0c8580 con 0x7fbf1c159fb0 2024-08-17T02:22:52.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0c8860 con 0x7fbf1c155820 2024-08-17T02:22:52.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf137fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbf1c159fb0 0x7fbf1c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:52.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf1bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbf1c159be0 0x7fbf1c0c7dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:60346/0 (socket says 172.21.15.100:60346) 2024-08-17T02:22:52.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf1bfff700 1 -- 172.21.15.100:0/3887032177 learned_addr learned my addr 172.21.15.100:0/3887032177 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:52.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf20ecb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 0x7fbf1c155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:52.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf137fe700 1 -- 172.21.15.100:0/3887032177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 msgr2=0x7fbf1c155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:52.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf137fe700 1 --2- 172.21.15.100:0/3887032177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 0x7fbf1c155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf137fe700 1 -- 172.21.15.100:0/3887032177 >> v1:172.21.15.123:6789/0 conn(0x7fbf1c159be0 legacy=0x7fbf1c0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:52.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf137fe700 1 -- 172.21.15.100:0/3887032177 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0c8b40 con 0x7fbf1c159fb0 2024-08-17T02:22:52.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf137fe700 1 --2- 172.21.15.100:0/3887032177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 0x7fbf1c15a430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf040098e0 tx=0x7fbf040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed80a9bffe121741 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:52.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf1b7fe700 1 -- 172.21.15.100:0/3887032177 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011630 con 0x7fbf1c159fb0 2024-08-17T02:22:52.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf1b7fe700 1 -- 172.21.15.100:0/3887032177 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf04011790 con 0x7fbf1c159fb0 2024-08-17T02:22:52.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf1b7fe700 1 -- 172.21.15.100:0/3887032177 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011b30 con 0x7fbf1c159fb0 2024-08-17T02:22:52.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 -- 172.21.15.100:0/3887032177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 msgr2=0x7fbf1c15a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:52.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 --2- 172.21.15.100:0/3887032177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 0x7fbf1c15a430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf040098e0 tx=0x7fbf040091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 -- 172.21.15.100:0/3887032177 shutdown_connections 2024-08-17T02:22:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.067+0000 7fbf216cc700 1 --2- 172.21.15.100:0/3887032177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 0x7fbf1c155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.071+0000 7fbf216cc700 1 --2- 172.21.15.100:0/3887032177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 0x7fbf1c15a430 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.071+0000 7fbf216cc700 1 -- 172.21.15.100:0/3887032177 >> 172.21.15.100:0/3887032177 conn(0x7fbf1c0c9830 msgr2=0x7fbf1c0c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.071+0000 7fbf216cc700 1 -- 172.21.15.100:0/3887032177 shutdown_connections 2024-08-17T02:22:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.071+0000 7fbf216cc700 1 -- 172.21.15.100:0/3887032177 wait complete. 2024-08-17T02:22:52.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.071+0000 7fbf216cc700 1 Processor -- start 2024-08-17T02:22:52.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.071+0000 7fbf216cc700 1 -- start start 2024-08-17T02:22:52.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf216cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 0x7fbf1c1694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:52.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf216cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf1c159be0 0x7fbf1c157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:52.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf216cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 0x7fbf1c1578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:52.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf216cc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1543a0 con 0x7fbf1c159be0 2024-08-17T02:22:52.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf216cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c169a00 con 0x7fbf1c159fb0 2024-08-17T02:22:52.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf216cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1562e0 con 0x7fbf1c155820 2024-08-17T02:22:52.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf1c159be0 0x7fbf1c157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:52.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf1c159be0 0x7fbf1c157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38590/0 (socket says 172.21.15.100:38590) 2024-08-17T02:22:52.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 -- 172.21.15.100:0/419423251 learned_addr learned my addr 172.21.15.100:0/419423251 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:52.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf1bfff700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 0x7fbf1c1694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:52.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf20ecb700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 0x7fbf1c1578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:52.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 -- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 msgr2=0x7fbf1c1694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:52.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 0x7fbf1c1694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 -- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 msgr2=0x7fbf1c1578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:52.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 0x7fbf1c1578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 -- 172.21.15.100:0/419423251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf04009420 con 0x7fbf1c159be0 2024-08-17T02:22:52.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf137fe700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf1c159be0 0x7fbf1c157380 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbf040030c0 tx=0x7fbf04016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:52.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf18ff9700 1 -- 172.21.15.100:0/419423251 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0401cd50 con 0x7fbf1c159be0 2024-08-17T02:22:52.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf18ff9700 1 -- 172.21.15.100:0/419423251 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0401a450 con 0x7fbf1c159be0 2024-08-17T02:22:52.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c1565c0 con 0x7fbf1c159be0 2024-08-17T02:22:52.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf18ff9700 1 -- 172.21.15.100:0/419423251 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0401a7f0 con 0x7fbf1c159be0 2024-08-17T02:22:52.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.075+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c156b80 con 0x7fbf1c159be0 2024-08-17T02:22:52.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.079+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c0638f0 con 0x7fbf1c159be0 2024-08-17T02:22:52.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.079+0000 7fbf18ff9700 1 -- 172.21.15.100:0/419423251 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbf0401ac20 con 0x7fbf1c159be0 2024-08-17T02:22:52.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.079+0000 7fbf18ff9700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbefc060860 0x7fbefc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:52.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.079+0000 7fbf18ff9700 1 -- 172.21.15.100:0/419423251 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fbf0407e5a0 con 0x7fbf1c159be0 2024-08-17T02:22:52.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.079+0000 7fbf1bfff700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbefc060860 0x7fbefc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:52.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.079+0000 7fbf1bfff700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbefc060860 0x7fbefc062d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0058d0 tx=0x7fbf0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:52.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.091+0000 7fbf18ff9700 1 -- 172.21.15.100:0/419423251 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbf0404ea50 con 0x7fbf1c159be0 2024-08-17T02:22:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.291+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbf1c0c9170 con 0x7fbf1c159be0 2024-08-17T02:22:52.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.927+0000 7fbf18ff9700 1 -- 172.21.15.100:0/419423251 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbf04052070 con 0x7fbf1c159be0 2024-08-17T02:22:52.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.939+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbefc060860 msgr2=0x7fbefc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:52.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.939+0000 7fbf216cc700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbefc060860 0x7fbefc062d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0058d0 tx=0x7fbf0c009040 comp rx=0 tx=0).stop 2024-08-17T02:22:52.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.939+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf1c159be0 msgr2=0x7fbf1c157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:52.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.939+0000 7fbf216cc700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf1c159be0 0x7fbf1c157380 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbf040030c0 tx=0x7fbf04016000 comp rx=0 tx=0).stop 2024-08-17T02:22:52.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.939+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 shutdown_connections 2024-08-17T02:22:52.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.943+0000 7fbf216cc700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbefc060860 0x7fbefc062d10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.943+0000 7fbf216cc700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1c159fb0 0x7fbf1c1578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.943+0000 7fbf216cc700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf1c159be0 0x7fbf1c157380 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.943+0000 7fbf216cc700 1 --2- 172.21.15.100:0/419423251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf1c155820 0x7fbf1c1694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:52.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.943+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 >> 172.21.15.100:0/419423251 conn(0x7fbf1c0c9830 msgr2=0x7fbf1c0ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:52.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.943+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 shutdown_connections 2024-08-17T02:22:52.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:52.943+0000 7fbf216cc700 1 -- 172.21.15.100:0/419423251 wait complete. 2024-08-17T02:22:52.952 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 4 and data pool 5 2024-08-17T02:22:52.967 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251b59e700 1 Processor -- start 2024-08-17T02:22:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251b59e700 1 -- start start 2024-08-17T02:22:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c1570d0 0x7f251c0cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251b59e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251c0af7d0 con 0x7f251c156d00 2024-08-17T02:22:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251c0aebc0 con 0x7f251c0cbe30 2024-08-17T02:22:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0ae9b0 con 0x7f251c1570d0 2024-08-17T02:22:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251a59c700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f251c156d00 0x7f251c0b1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48938/0 (socket says 172.21.15.100:48938) 2024-08-17T02:22:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251a59c700 1 -- 172.21.15.100:0/737175685 learned_addr learned my addr 172.21.15.100:0/737175685 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f2519d9b700 1 --2- 172.21.15.100:0/737175685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c1570d0 0x7f251c0cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251959a700 1 -- 172.21.15.100:0/737175685 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182407135 0 0) 0x7f251c0af7d0 con 0x7f251c156d00 2024-08-17T02:22:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251959a700 1 -- 172.21.15.100:0/737175685 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2504003570 con 0x7f251c156d00 2024-08-17T02:22:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251959a700 1 -- 172.21.15.100:0/737175685 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136648814 0 0) 0x7f251c0aebc0 con 0x7f251c0cbe30 2024-08-17T02:22:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251959a700 1 -- 172.21.15.100:0/737175685 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f251c0af7d0 con 0x7f251c0cbe30 2024-08-17T02:22:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f2519d9b700 1 -- 172.21.15.100:0/737175685 >> v1:172.21.15.123:6789/0 conn(0x7f251c0cbe30 legacy=0x7f251c0afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f2519d9b700 1 -- 172.21.15.100:0/737175685 >> v1:172.21.15.100:6789/0 conn(0x7f251c156d00 legacy=0x7f251c0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f2519d9b700 1 -- 172.21.15.100:0/737175685 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c0aeea0 con 0x7f251c1570d0 2024-08-17T02:22:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f2519d9b700 1 --2- 172.21.15.100:0/737175685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c1570d0 0x7f251c0cb8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2514009e90 tx=0x7f2514009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4532221727f0ed39 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251959a700 1 -- 172.21.15.100:0/737175685 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251400e070 con 0x7f251c1570d0 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251959a700 1 -- 172.21.15.100:0/737175685 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251400bb80 con 0x7f251c1570d0 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.063+0000 7f251959a700 1 -- 172.21.15.100:0/737175685 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514012430 con 0x7f251c1570d0 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- 172.21.15.100:0/737175685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c1570d0 msgr2=0x7f251c0cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 --2- 172.21.15.100:0/737175685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c1570d0 0x7f251c0cb8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2514009e90 tx=0x7f2514009780 comp rx=0 tx=0).stop 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- 172.21.15.100:0/737175685 shutdown_connections 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 --2- 172.21.15.100:0/737175685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c1570d0 0x7f251c0cb8f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- 172.21.15.100:0/737175685 >> 172.21.15.100:0/737175685 conn(0x7f251c0cdeb0 msgr2=0x7f251c0ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- 172.21.15.100:0/737175685 shutdown_connections 2024-08-17T02:22:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- 172.21.15.100:0/737175685 wait complete. 2024-08-17T02:22:53.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 Processor -- start 2024-08-17T02:22:53.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- start start 2024-08-17T02:22:53.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c0cbe30 0x7f251c06a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f251c156d00 0x7f251c14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f251c1570d0 0x7f251c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c1552f0 con 0x7f251c156d00 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c154720 con 0x7f251c1570d0 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c1549c0 con 0x7f251c0cbe30 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f251c156d00 0x7f251c14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f251c156d00 0x7f251c14acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38594/0 (socket says 172.21.15.100:38594) 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 -- 172.21.15.100:0/3678703400 learned_addr learned my addr 172.21.15.100:0/3678703400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251a59c700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c0cbe30 0x7f251c06a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251ad9d700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f251c1570d0 0x7f251c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 -- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c0cbe30 msgr2=0x7f251c06a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c0cbe30 0x7f251c06a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 -- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f251c1570d0 msgr2=0x7f251c1513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f251c1570d0 0x7f251c1513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 -- 172.21.15.100:0/3678703400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514009a20 con 0x7f251c156d00 2024-08-17T02:22:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f2519d9b700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f251c156d00 0x7f251c14acc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f25140099f0 tx=0x7f2514019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f250a7fc700 1 -- 172.21.15.100:0/3678703400 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514012df0 con 0x7f251c156d00 2024-08-17T02:22:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c154ca0 con 0x7f251c156d00 2024-08-17T02:22:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c06a750 con 0x7f251c156d00 2024-08-17T02:22:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f250a7fc700 1 -- 172.21.15.100:0/3678703400 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2514027470 con 0x7f251c156d00 2024-08-17T02:22:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.067+0000 7f250a7fc700 1 -- 172.21.15.100:0/3678703400 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514027830 con 0x7f251c156d00 2024-08-17T02:22:53.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.071+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c0638f0 con 0x7f251c156d00 2024-08-17T02:22:53.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.071+0000 7f250a7fc700 1 -- 172.21.15.100:0/3678703400 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2514024070 con 0x7f251c156d00 2024-08-17T02:22:53.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.071+0000 7f250a7fc700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24fc060860 0x7f24fc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.071+0000 7f250a7fc700 1 -- 172.21.15.100:0/3678703400 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f251407f6d0 con 0x7f251c156d00 2024-08-17T02:22:53.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.071+0000 7f251a59c700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24fc060860 0x7f24fc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.071+0000 7f251a59c700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24fc060860 0x7f24fc062d10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25100058d0 tx=0x7f2510009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.079+0000 7f250a7fc700 1 -- 172.21.15.100:0/3678703400 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f251404fb20 con 0x7f251c156d00 2024-08-17T02:22:53.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.243+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f251c0711a0 con 0x7f251c156d00 2024-08-17T02:22:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.247+0000 7f250a7fc700 1 -- 172.21.15.100:0/3678703400 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f251402c090 con 0x7f251c156d00 2024-08-17T02:22:53.251 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:53.251 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":32,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:52.295350+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:50.544046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:51.275880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:53.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24fc060860 msgr2=0x7f24fc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24fc060860 0x7f24fc062d10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25100058d0 tx=0x7f2510009040 comp rx=0 tx=0).stop 2024-08-17T02:22:53.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f251c156d00 msgr2=0x7f251c14acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f251c156d00 0x7f251c14acc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f25140099f0 tx=0x7f2514019000 comp rx=0 tx=0).stop 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 shutdown_connections 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f24fc060860 0x7f24fc062d10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f251c1570d0 0x7f251c1513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f251c156d00 0x7f251c14acc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 --2- 172.21.15.100:0/3678703400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251c0cbe30 0x7f251c06a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 >> 172.21.15.100:0/3678703400 conn(0x7f251c0cdeb0 msgr2=0x7f251c0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 shutdown_connections 2024-08-17T02:22:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.251+0000 7f251b59e700 1 -- 172.21.15.100:0/3678703400 wait complete. 2024-08-17T02:22:53.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:53.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2eb54700 1 Processor -- start 2024-08-17T02:22:53.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2eb54700 1 -- start start 2024-08-17T02:22:53.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2eb54700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a280cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2eb54700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0a280d0010 0x7f0a280d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2eb54700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a280cf190 con 0x7f0a2815cea0 2024-08-17T02:22:53.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2eb54700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a280d09a0 con 0x7f0a280d0010 2024-08-17T02:22:53.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2eb54700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280ce5e0 con 0x7f0a28159040 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2d351700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0a280d0010 0x7f0a280d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2e353700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0a2815cea0 0x7f0a2815d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48950/0 (socket says 172.21.15.100:48950) 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2d351700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0a280d0010 0x7f0a280d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38606/0 (socket says 172.21.15.100:38606) 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2e353700 1 -- 172.21.15.100:0/2847053300 learned_addr learned my addr 172.21.15.100:0/2847053300 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2db52700 1 --2- 172.21.15.100:0/2847053300 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a280cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2cb50700 1 -- 172.21.15.100:0/2847053300 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608451545 0 0) 0x7f0a280cf190 con 0x7f0a2815cea0 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2cb50700 1 -- 172.21.15.100:0/2847053300 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a14003570 con 0x7f0a2815cea0 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2d351700 1 -- 172.21.15.100:0/2847053300 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 msgr2=0x7f0a280cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2d351700 1 --2- 172.21.15.100:0/2847053300 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a280cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2d351700 1 -- 172.21.15.100:0/2847053300 >> v1:172.21.15.100:6789/0 conn(0x7f0a2815cea0 legacy=0x7f0a2815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2d351700 1 -- 172.21.15.100:0/2847053300 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280ce860 con 0x7f0a280d0010 2024-08-17T02:22:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2d351700 1 --2- 172.21.15.100:0/2847053300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a280d0010 0x7f0a280d0460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0a100098e0 tx=0x7f0a100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7134f6e053a5fac3 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2cb50700 1 -- 172.21.15.100:0/2847053300 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10011630 con 0x7f0a280d0010 2024-08-17T02:22:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2cb50700 1 -- 172.21.15.100:0/2847053300 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10011790 con 0x7f0a280d0010 2024-08-17T02:22:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.367+0000 7f0a2cb50700 1 -- 172.21.15.100:0/2847053300 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10011b30 con 0x7f0a280d0010 2024-08-17T02:22:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- 172.21.15.100:0/2847053300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a280d0010 msgr2=0x7f0a280d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/2847053300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a280d0010 0x7f0a280d0460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0a100098e0 tx=0x7f0a100091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- 172.21.15.100:0/2847053300 shutdown_connections 2024-08-17T02:22:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/2847053300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a280d0010 0x7f0a280d0460 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/2847053300 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a280cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- 172.21.15.100:0/2847053300 >> 172.21.15.100:0/2847053300 conn(0x7f0a28068250 msgr2=0x7f0a28068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- 172.21.15.100:0/2847053300 shutdown_connections 2024-08-17T02:22:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- 172.21.15.100:0/2847053300 wait complete. 2024-08-17T02:22:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 Processor -- start 2024-08-17T02:22:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- start start 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a280d0010 0x7f0a280a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a2815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a2815cea0 0x7f0a281516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2816a4e0 con 0x7f0a2815cea0 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a281698d0 con 0x7f0a280d0010 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2eb54700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28169bb0 con 0x7f0a28159040 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2e353700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a2815cea0 0x7f0a281516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a2815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a2815c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:46014/0 (socket says 172.21.15.100:46014) 2024-08-17T02:22:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 -- 172.21.15.100:0/3739357548 learned_addr learned my addr 172.21.15.100:0/3739357548 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2db52700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a280d0010 0x7f0a280a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 -- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a280d0010 msgr2=0x7f0a280a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a280d0010 0x7f0a280a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 -- 172.21.15.100:0/3739357548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a2815cea0 msgr2=0x7f0a281516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a2815cea0 0x7f0a281516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 -- 172.21.15.100:0/3739357548 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a10009420 con 0x7f0a28159040 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2e353700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a2815cea0 0x7f0a281516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.371+0000 7f0a2d351700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a2815c100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0a28088ef0 tx=0x7f0a10017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.375+0000 7f0a25ffb700 1 -- 172.21.15.100:0/3739357548 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10000d10 con 0x7f0a28159040 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.375+0000 7f0a25ffb700 1 -- 172.21.15.100:0/3739357548 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10011b10 con 0x7f0a28159040 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.375+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a28169e90 con 0x7f0a28159040 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.375+0000 7f0a25ffb700 1 -- 172.21.15.100:0/3739357548 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10025660 con 0x7f0a28159040 2024-08-17T02:22:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.375+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a280a6ef0 con 0x7f0a28159040 2024-08-17T02:22:53.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.375+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a280cf340 con 0x7f0a28159040 2024-08-17T02:22:53.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.375+0000 7f0a25ffb700 1 -- 172.21.15.100:0/3739357548 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0a10046b20 con 0x7f0a28159040 2024-08-17T02:22:53.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.379+0000 7f0a25ffb700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a08060860 0x7f0a08062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.379+0000 7f0a25ffb700 1 -- 172.21.15.100:0/3739357548 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0a1007e2f0 con 0x7f0a28159040 2024-08-17T02:22:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.379+0000 7f0a2db52700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a08060860 0x7f0a08062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.379+0000 7f0a2db52700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a08060860 0x7f0a08062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800f570 tx=0x7f0a1800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.387+0000 7f0a25ffb700 1 -- 172.21.15.100:0/3739357548 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0a1004e800 con 0x7f0a28159040 2024-08-17T02:22:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.555+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a280cf340 con 0x7f0a28159040 2024-08-17T02:22:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.555+0000 7f0a25ffb700 1 -- 172.21.15.100:0/3739357548 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f0a1004dc50 con 0x7f0a28159040 2024-08-17T02:22:53.560 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:53.560 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":32,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:52.295350+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:50.544046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:51.275880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a08060860 msgr2=0x7f0a08062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a08060860 0x7f0a08062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800f570 tx=0x7f0a1800b040 comp rx=0 tx=0).stop 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 msgr2=0x7f0a2815c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a2815c100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0a28088ef0 tx=0x7f0a10017000 comp rx=0 tx=0).stop 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 shutdown_connections 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0a08060860 0x7f0a08062d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a2815cea0 0x7f0a281516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a28159040 0x7f0a2815c100 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 --2- 172.21.15.100:0/3739357548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a280d0010 0x7f0a280a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 >> 172.21.15.100:0/3739357548 conn(0x7f0a28068250 msgr2=0x7f0a28159410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 shutdown_connections 2024-08-17T02:22:53.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.559+0000 7f0a2eb54700 1 -- 172.21.15.100:0/3739357548 wait complete. 2024-08-17T02:22:53.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-17T02:22:53.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fda521700 1 Processor -- start 2024-08-17T02:22:53.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fda521700 1 -- start start 2024-08-17T02:22:53.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fda521700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2fd41590b0 0x7f2fd4066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fda521700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd40cd2b0 0x7f2fd4067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fda521700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd40ce210 con 0x7f2fd4159480 2024-08-17T02:22:53.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fda521700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd40cba00 con 0x7f2fd41590b0 2024-08-17T02:22:53.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fda521700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40cdc90 con 0x7f2fd40cd2b0 2024-08-17T02:22:53.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fd951f700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2fd41590b0 0x7f2fd4066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fd951f700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2fd41590b0 0x7f2fd4066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38618/0 (socket says 172.21.15.100:38618) 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.679+0000 7f2fd951f700 1 -- 172.21.15.100:0/2735604985 learned_addr learned my addr 172.21.15.100:0/2735604985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd9d20700 1 --2- 172.21.15.100:0/2735604985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd40cd2b0 0x7f2fd4067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd951f700 1 -- 172.21.15.100:0/2735604985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd40cd2b0 msgr2=0x7f2fd4067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd951f700 1 --2- 172.21.15.100:0/2735604985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd40cd2b0 0x7f2fd4067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd951f700 1 -- 172.21.15.100:0/2735604985 >> v1:172.21.15.123:6789/0 conn(0x7f2fd4159480 legacy=0x7f2fd415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd951f700 1 -- 172.21.15.100:0/2735604985 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40cb440 con 0x7f2fd41590b0 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd951f700 1 --2- 172.21.15.100:0/2735604985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd4066e00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40099b0 tx=0x7f2fc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8639fc0278e276a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd3fff700 1 -- 172.21.15.100:0/2735604985 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400b820 con 0x7f2fd41590b0 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd3fff700 1 -- 172.21.15.100:0/2735604985 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc400b980 con 0x7f2fd41590b0 2024-08-17T02:22:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd3fff700 1 -- 172.21.15.100:0/2735604985 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400bd20 con 0x7f2fd41590b0 2024-08-17T02:22:53.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- 172.21.15.100:0/2735604985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 msgr2=0x7f2fd4066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 --2- 172.21.15.100:0/2735604985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd4066e00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40099b0 tx=0x7f2fc4009250 comp rx=0 tx=0).stop 2024-08-17T02:22:53.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- 172.21.15.100:0/2735604985 shutdown_connections 2024-08-17T02:22:53.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 --2- 172.21.15.100:0/2735604985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd40cd2b0 0x7f2fd4067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 --2- 172.21.15.100:0/2735604985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd4066e00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- 172.21.15.100:0/2735604985 >> 172.21.15.100:0/2735604985 conn(0x7f2fd40d0b90 msgr2=0x7f2fd40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- 172.21.15.100:0/2735604985 shutdown_connections 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- 172.21.15.100:0/2735604985 wait complete. 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 Processor -- start 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- start start 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd40cd2b0 0x7f2fd4143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd40c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd4159480 0x7f2fd40c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40b4920 con 0x7f2fd41590b0 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40b5780 con 0x7f2fd40cd2b0 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fda521700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40b4b70 con 0x7f2fd4159480 2024-08-17T02:22:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd8d1e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd40c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd8d1e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd40c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38632/0 (socket says 172.21.15.100:38632) 2024-08-17T02:22:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd8d1e700 1 -- 172.21.15.100:0/3340694510 learned_addr learned my addr 172.21.15.100:0/3340694510 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd951f700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd40cd2b0 0x7f2fd4143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.683+0000 7f2fd9d20700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd4159480 0x7f2fd40c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd8d1e700 1 -- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd4159480 msgr2=0x7f2fd40c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd8d1e700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd4159480 0x7f2fd40c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd8d1e700 1 -- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd40cd2b0 msgr2=0x7f2fd4143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd8d1e700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd40cd2b0 0x7f2fd4143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd8d1e700 1 -- 172.21.15.100:0/3340694510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc40094f0 con 0x7f2fd41590b0 2024-08-17T02:22:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd8d1e700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd40c7c60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b5e0 tx=0x7f2fc8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd17fa700 1 -- 172.21.15.100:0/3340694510 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800f510 con 0x7f2fd41590b0 2024-08-17T02:22:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd17fa700 1 -- 172.21.15.100:0/3340694510 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc800f670 con 0x7f2fd41590b0 2024-08-17T02:22:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd40b4e50 con 0x7f2fd41590b0 2024-08-17T02:22:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd17fa700 1 -- 172.21.15.100:0/3340694510 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800fa30 con 0x7f2fd41590b0 2024-08-17T02:22:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd40b5410 con 0x7f2fd41590b0 2024-08-17T02:22:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd40b58f0 con 0x7f2fd41590b0 2024-08-17T02:22:53.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.687+0000 7f2fd17fa700 1 -- 172.21.15.100:0/3340694510 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2fc801a400 con 0x7f2fd41590b0 2024-08-17T02:22:53.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.691+0000 7f2fd17fa700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2fc0060860 0x7f2fc0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:53.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.691+0000 7f2fd17fa700 1 -- 172.21.15.100:0/3340694510 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fc807c5e0 con 0x7f2fd41590b0 2024-08-17T02:22:53.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.691+0000 7f2fd951f700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2fc0060860 0x7f2fc0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.691+0000 7f2fd951f700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2fc0060860 0x7f2fc0062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40099b0 tx=0x7f2fc4016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:53.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.695+0000 7f2fd17fa700 1 -- 172.21.15.100:0/3340694510 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2fc804cab0 con 0x7f2fd41590b0 2024-08-17T02:22:53.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.895+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2fd4064330 con 0x7f2fd41590b0 2024-08-17T02:22:53.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.943+0000 7f2fd17fa700 1 -- 172.21.15.100:0/3340694510 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2fc80500d0 con 0x7f2fd41590b0 2024-08-17T02:22:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2fc0060860 msgr2=0x7f2fc0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2fc0060860 0x7f2fc0062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40099b0 tx=0x7f2fc4016000 comp rx=0 tx=0).stop 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 msgr2=0x7f2fd40c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd40c7c60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b5e0 tx=0x7f2fc8009120 comp rx=0 tx=0).stop 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 shutdown_connections 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2fc0060860 0x7f2fc0062d10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fd4159480 0x7f2fd40c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd41590b0 0x7f2fd40c7c60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 --2- 172.21.15.100:0/3340694510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd40cd2b0 0x7f2fd4143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 >> 172.21.15.100:0/3340694510 conn(0x7f2fd40d0b90 msgr2=0x7f2fd40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 shutdown_connections 2024-08-17T02:22:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:53.947+0000 7f2fda521700 1 -- 172.21.15.100:0/3340694510 wait complete. 2024-08-17T02:22:53.966 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-17T02:22:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.071+0000 7f76d7969700 1 Processor -- start 2024-08-17T02:22:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.071+0000 7f76d7969700 1 -- start start 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.071+0000 7f76d7969700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d00cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.071+0000 7f76d7969700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f76d015d5b0 0x7f76d0163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.071+0000 7f76d7969700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d00c9df0 con 0x7f76d00cc170 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.071+0000 7f76d7969700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d00cdca0 con 0x7f76d015d5b0 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.071+0000 7f76d7969700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d00c91e0 con 0x7f76d0153660 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5f06700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f76d015d5b0 0x7f76d0163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5705700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d00cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d4f04700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f76d00cc170 0x7f76d015cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:60378/0 (socket says 172.21.15.100:60378) 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5f06700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f76d015d5b0 0x7f76d0163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:50288/0 (socket says 172.21.15.100:50288) 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d4f04700 1 -- 172.21.15.100:0/1480087706 learned_addr learned my addr 172.21.15.100:0/1480087706 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5705700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d00cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:46046/0 (socket says 172.21.15.100:46046) 2024-08-17T02:22:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5f06700 1 -- 172.21.15.100:0/1480087706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 msgr2=0x7f76d00cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5f06700 1 --2- 172.21.15.100:0/1480087706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d00cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5f06700 1 -- 172.21.15.100:0/1480087706 >> v1:172.21.15.123:6789/0 conn(0x7f76d00cc170 legacy=0x7f76d015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5f06700 1 -- 172.21.15.100:0/1480087706 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d00c94c0 con 0x7f76d015d5b0 2024-08-17T02:22:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d5f06700 1 --2- 172.21.15.100:0/1480087706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d0163030 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009e60 tx=0x7f76c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f4492a02111432 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76c3fff700 1 -- 172.21.15.100:0/1480087706 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c800e070 con 0x7f76d015d5b0 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76c3fff700 1 -- 172.21.15.100:0/1480087706 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c800bb60 con 0x7f76d015d5b0 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76c3fff700 1 -- 172.21.15.100:0/1480087706 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8012430 con 0x7f76d015d5b0 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d7969700 1 -- 172.21.15.100:0/1480087706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 msgr2=0x7f76d0163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d7969700 1 --2- 172.21.15.100:0/1480087706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d0163030 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009e60 tx=0x7f76c8009780 comp rx=0 tx=0).stop 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d7969700 1 -- 172.21.15.100:0/1480087706 shutdown_connections 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d7969700 1 --2- 172.21.15.100:0/1480087706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d0163030 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d7969700 1 --2- 172.21.15.100:0/1480087706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d00cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d7969700 1 -- 172.21.15.100:0/1480087706 >> 172.21.15.100:0/1480087706 conn(0x7f76d00caad0 msgr2=0x7f76d00caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d7969700 1 -- 172.21.15.100:0/1480087706 shutdown_connections 2024-08-17T02:22:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.075+0000 7f76d7969700 1 -- 172.21.15.100:0/1480087706 wait complete. 2024-08-17T02:22:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 Processor -- start 2024-08-17T02:22:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 -- start start 2024-08-17T02:22:54.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d00cc170 0x7f76d0155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:54.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d01519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0155f70 con 0x7f76d00cc170 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0155d80 con 0x7f76d015d5b0 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0151f10 con 0x7f76d0153660 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5705700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d00cc170 0x7f76d0155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d4f04700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d01519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d015c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:50296/0 (socket says 172.21.15.100:50296) 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5705700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d00cc170 0x7f76d0155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38644/0 (socket says 172.21.15.100:38644) 2024-08-17T02:22:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d4f04700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d01519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:46062/0 (socket says 172.21.15.100:46062) 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 -- 172.21.15.100:0/3382412154 learned_addr learned my addr 172.21.15.100:0/3382412154 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 -- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 msgr2=0x7f76d01519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d01519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 -- 172.21.15.100:0/3382412154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d00cc170 msgr2=0x7f76d0155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d00cc170 0x7f76d0155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 -- 172.21.15.100:0/3382412154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c8009a20 con 0x7f76d015d5b0 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d4f04700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d01519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5705700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d00cc170 0x7f76d0155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d5f06700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d015c050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76c8013000 tx=0x7f76c8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76c17fa700 1 -- 172.21.15.100:0/3382412154 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8012ea0 con 0x7f76d015d5b0 2024-08-17T02:22:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76c17fa700 1 -- 172.21.15.100:0/3382412154 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c8027470 con 0x7f76d015d5b0 2024-08-17T02:22:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d00b1000 con 0x7f76d015d5b0 2024-08-17T02:22:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76c17fa700 1 -- 172.21.15.100:0/3382412154 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8027850 con 0x7f76d015d5b0 2024-08-17T02:22:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.079+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d00b15c0 con 0x7f76d015d5b0 2024-08-17T02:22:54.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.083+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d00645c0 con 0x7f76d015d5b0 2024-08-17T02:22:54.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.083+0000 7f76c17fa700 1 -- 172.21.15.100:0/3382412154 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f76c8024070 con 0x7f76d015d5b0 2024-08-17T02:22:54.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.083+0000 7f76c17fa700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f76b4060860 0x7f76b4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:54.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.083+0000 7f76c17fa700 1 -- 172.21.15.100:0/3382412154 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f76c807f6e0 con 0x7f76d015d5b0 2024-08-17T02:22:54.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.083+0000 7f76d5705700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f76b4060860 0x7f76b4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:54.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.087+0000 7f76d5705700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f76b4060860 0x7f76b4062d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f76cc011480 tx=0x7f76cc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.095+0000 7f76c17fa700 1 -- 172.21.15.100:0/3382412154 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f76c804fa90 con 0x7f76d015d5b0 2024-08-17T02:22:54.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.299+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f76d00c4bc0 con 0x7f76d015d5b0 2024-08-17T02:22:54.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.951+0000 7f76c17fa700 1 -- 172.21.15.100:0/3382412154 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f76c80530b0 con 0x7f76d015d5b0 2024-08-17T02:22:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f76b4060860 msgr2=0x7f76b4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f76b4060860 0x7f76b4062d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f76cc011480 tx=0x7f76cc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 msgr2=0x7f76d015c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d015c050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76c8013000 tx=0x7f76c8019000 comp rx=0 tx=0).stop 2024-08-17T02:22:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 shutdown_connections 2024-08-17T02:22:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f76b4060860 0x7f76b4062d10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d015d5b0 0x7f76d015c050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d0153660 0x7f76d01519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 --2- 172.21.15.100:0/3382412154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d00cc170 0x7f76d0155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 >> 172.21.15.100:0/3382412154 conn(0x7f76d00caad0 msgr2=0x7f76d00cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 shutdown_connections 2024-08-17T02:22:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:54.955+0000 7f76d7969700 1 -- 172.21.15.100:0/3382412154 wait complete. 2024-08-17T02:22:54.974 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:55.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a6fab700 1 Processor -- start 2024-08-17T02:22:55.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a6fab700 1 -- start start 2024-08-17T02:22:55.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a6fab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a015d540 0x7f60a00af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a6fab700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a00b0270 con 0x7f60a006e830 2024-08-17T02:22:55.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a6fab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a00af660 con 0x7f60a00a9430 2024-08-17T02:22:55.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a6fab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00af940 con 0x7f60a015d540 2024-08-17T02:22:55.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609ffff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f60a006e830 0x7f60a00a8d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45802/0 (socket says 172.21.15.100:45802) 2024-08-17T02:22:55.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609ffff700 1 -- 172.21.15.100:0/1892444071 learned_addr learned my addr 172.21.15.100:0/1892444071 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:55.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a4d47700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a015d540 0x7f60a00af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438044481 0 0) 0x7f60a00b0270 con 0x7f60a006e830 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6088003570 con 0x7f60a006e830 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895161778 0 0) 0x7f6088003570 con 0x7f60a006e830 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a015d540 msgr2=0x7f60a00af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a015d540 0x7f60a00af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 >> v1:172.21.15.123:6789/0 conn(0x7f60a00a9430 legacy=0x7f60a00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a00afc20 con 0x7f60a006e830 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f60980017f0 con 0x7f60a006e830 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 >> v1:172.21.15.100:6789/0 conn(0x7f60a006e830 legacy=0x7f60a00a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6088003a90 0x7f6088003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a015d540 0x7f6088004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6088004d50 0x7f60880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00af940 con 0x7f6088004d50 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608800bb70 con 0x7f6088003a90 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608800bd30 con 0x7f60a015d540 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a5548700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6088004d50 0x7f60880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a4d47700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6088003a90 0x7f6088003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f609ffff700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a015d540 0x7f6088004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a5548700 1 -- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a015d540 msgr2=0x7f6088004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a5548700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a015d540 0x7f6088004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a5548700 1 -- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6088003a90 msgr2=0x7f6088003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a5548700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6088003a90 0x7f6088003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.079+0000 7f60a5548700 1 -- 172.21.15.100:0/1892444071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6088003570 con 0x7f6088004d50 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f60a5548700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6088004d50 0x7f60880051c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f60900018b0 tx=0x7f6090007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71f9a4a1b1a471 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090012420 con 0x7f6088004d50 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6090012580 con 0x7f6088004d50 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f609f7fe700 1 -- 172.21.15.100:0/1892444071 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090012920 con 0x7f6088004d50 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f60a6fab700 1 -- 172.21.15.100:0/1892444071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6088004d50 msgr2=0x7f60880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f60a6fab700 1 --2- 172.21.15.100:0/1892444071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6088004d50 0x7f60880051c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f60900018b0 tx=0x7f6090007d30 comp rx=0 tx=0).stop 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f60a5548700 1 -- 172.21.15.100:0/1892444071 reap_dead start 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f60a6fab700 1 -- 172.21.15.100:0/1892444071 shutdown_connections 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f60a6fab700 1 -- 172.21.15.100:0/1892444071 >> 172.21.15.100:0/1892444071 conn(0x7f60a0066e00 msgr2=0x7f60a0067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:55.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f60a6fab700 1 -- 172.21.15.100:0/1892444071 shutdown_connections 2024-08-17T02:22:55.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.083+0000 7f60a6fab700 1 -- 172.21.15.100:0/1892444071 wait complete. 2024-08-17T02:22:55.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 Processor -- start 2024-08-17T02:22:55.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 -- start start 2024-08-17T02:22:55.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a0156070 0x7f60a0162610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60a0156440 0x7f60a0162b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a0163090 0x7f60a00aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a0163580 con 0x7f60a0163090 2024-08-17T02:22:55.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a0097a10 con 0x7f60a0156440 2024-08-17T02:22:55.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00970e0 con 0x7f60a0156070 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a0163090 0x7f60a00aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a0163090 0x7f60a00aab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58760/0 (socket says 172.21.15.100:58760) 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 -- 172.21.15.100:0/1010183630 learned_addr learned my addr 172.21.15.100:0/1010183630 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f609ffff700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60a0156440 0x7f60a0162b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a4d47700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a0156070 0x7f60a0162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 -- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a0156070 msgr2=0x7f60a0162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a0156070 0x7f60a0162610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 -- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60a0156440 msgr2=0x7f60a0162b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60a0156440 0x7f60a0162b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 -- 172.21.15.100:0/1010183630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a00973c0 con 0x7f60a0163090 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a5548700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a0163090 0x7f60a00aab90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f609000c3f0 tx=0x7f6090015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f609cff9700 1 -- 172.21.15.100:0/1010183630 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090012420 con 0x7f60a0163090 2024-08-17T02:22:55.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f609cff9700 1 -- 172.21.15.100:0/1010183630 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6090001200 con 0x7f60a0163090 2024-08-17T02:22:55.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f609cff9700 1 -- 172.21.15.100:0/1010183630 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60900126f0 con 0x7f60a0163090 2024-08-17T02:22:55.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a00976a0 con 0x7f60a0163090 2024-08-17T02:22:55.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.087+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a0096c10 con 0x7f60a0163090 2024-08-17T02:22:55.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.091+0000 7f609cff9700 1 -- 172.21.15.100:0/1010183630 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6090019070 con 0x7f60a0163090 2024-08-17T02:22:55.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.091+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a0096e00 con 0x7f60a0163090 2024-08-17T02:22:55.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.091+0000 7f609cff9700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f608c060890 0x7f608c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.091+0000 7f609cff9700 1 -- 172.21.15.100:0/1010183630 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f609007db50 con 0x7f60a0163090 2024-08-17T02:22:55.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.091+0000 7f60a4d47700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f608c060890 0x7f608c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.091+0000 7f60a4d47700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f608c060890 0x7f608c062d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6094006f00 tx=0x7f6094008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.099+0000 7f609cff9700 1 -- 172.21.15.100:0/1010183630 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f609004dfc0 con 0x7f60a0163090 2024-08-17T02:22:55.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.303+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60a0097ba0 con 0x7f60a0163090 2024-08-17T02:22:55.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f609cff9700 1 -- 172.21.15.100:0/1010183630 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f6090017020 con 0x7f60a0163090 2024-08-17T02:22:55.309 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:55.309 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:52.296101+0000","modified":"2024-08-17T02:22:54.942750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4980":{"gid":4980,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3037167008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3037167008},{"type":"v1","addr":"172.21.15.100:6835","nonce":3037167008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f608c060890 msgr2=0x7f608c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f608c060890 0x7f608c062d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6094006f00 tx=0x7f6094008040 comp rx=0 tx=0).stop 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a0163090 msgr2=0x7f60a00aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a0163090 0x7f60a00aab90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f609000c3f0 tx=0x7f6090015000 comp rx=0 tx=0).stop 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 shutdown_connections 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f608c060890 0x7f608c062d40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a0163090 0x7f60a00aab90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60a0156440 0x7f60a0162b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 --2- 172.21.15.100:0/1010183630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60a0156070 0x7f60a0162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 >> 172.21.15.100:0/1010183630 conn(0x7f60a0066e00 msgr2=0x7f60a0159010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 shutdown_connections 2024-08-17T02:22:55.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.307+0000 7f60a6fab700 1 -- 172.21.15.100:0/1010183630 wait complete. 2024-08-17T02:22:55.313 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 13 2024-08-17T02:22:55.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:22:55.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.427+0000 7f5e40a77700 1 Processor -- start 2024-08-17T02:22:55.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.427+0000 7f5e40a77700 1 -- start start 2024-08-17T02:22:55.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5e3c154900 0x7f5e3c0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0b1730 con 0x7f5e3c0b12d0 2024-08-17T02:22:55.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0cbaa0 con 0x7f5e3c0cb560 2024-08-17T02:22:55.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c068790 con 0x7f5e3c154900 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5e3c0b12d0 0x7f5e3c0cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45808/0 (socket says 172.21.15.100:45808) 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 -- 172.21.15.100:0/2654158044 learned_addr learned my addr 172.21.15.100:0/2654158044 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087209834 0 0) 0x7f5e24001110 con 0x7f5e3c0b12d0 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e28003570 con 0x7f5e3c0b12d0 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3b7fe700 1 --2- 172.21.15.100:0/2654158044 >> v2:172.21.15.123:3300/0 conn(0x7f5e3c154900 0x7f5e3c0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4164361862 0 0) 0x7f5e28003570 con 0x7f5e3c0b12d0 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 >> v2:172.21.15.123:3300/0 conn(0x7f5e3c154900 msgr2=0x7f5e3c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 --2- 172.21.15.100:0/2654158044 >> v2:172.21.15.123:3300/0 conn(0x7f5e3c154900 0x7f5e3c0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 >> v1:172.21.15.123:6789/0 conn(0x7f5e3c0cb560 legacy=0x7f5e3c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c157630 con 0x7f5e3c0b12d0 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f5e24001600 con 0x7f5e3c0b12d0 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3b7fe700 1 --2- 172.21.15.100:0/2654158044 >> v2:172.21.15.123:3300/0 conn(0x7f5e3c154900 0x7f5e3c0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 >> v1:172.21.15.100:6789/0 conn(0x7f5e3c0b12d0 legacy=0x7f5e3c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e28003a60 0x7f5e28003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28004810 0x7f5e28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e280051c0 0x7f5e2800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c068790 con 0x7f5e28004810 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28004810 0x7f5e28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2800c0e0 con 0x7f5e28003a60 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2800c2a0 con 0x7f5e280051c0 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3b7fe700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e28003a60 0x7f5e28003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3bfff700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e280051c0 0x7f5e2800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 -- 172.21.15.100:0/2654158044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e280051c0 msgr2=0x7f5e2800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e280051c0 0x7f5e2800ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 -- 172.21.15.100:0/2654158044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e28003a60 msgr2=0x7f5e28003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e28003a60 0x7f5e28003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 -- 172.21.15.100:0/2654158044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28003570 con 0x7f5e28004810 2024-08-17T02:22:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3bfff700 1 -- 172.21.15.100:0/2654158044 reap_dead start 2024-08-17T02:22:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3affd700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28004810 0x7f5e28004c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400fba0 tx=0x7f5e24008dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf9f86580612ff8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24014070 con 0x7f5e28004810 2024-08-17T02:22:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24005920 con 0x7f5e28004810 2024-08-17T02:22:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e3a7fc700 1 -- 172.21.15.100:0/2654158044 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24005cc0 con 0x7f5e28004810 2024-08-17T02:22:55.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 -- 172.21.15.100:0/2654158044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28004810 msgr2=0x7f5e28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28004810 0x7f5e28004c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400fba0 tx=0x7f5e24008dc0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 -- 172.21.15.100:0/2654158044 shutdown_connections 2024-08-17T02:22:55.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 --2- 172.21.15.100:0/2654158044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28004810 0x7f5e28004c80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 -- 172.21.15.100:0/2654158044 >> 172.21.15.100:0/2654158044 conn(0x7f5e3c0678d0 msgr2=0x7f5e3c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:55.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 -- 172.21.15.100:0/2654158044 shutdown_connections 2024-08-17T02:22:55.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.431+0000 7f5e40a77700 1 -- 172.21.15.100:0/2654158044 wait complete. 2024-08-17T02:22:55.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e40a77700 1 Processor -- start 2024-08-17T02:22:55.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e40a77700 1 -- start start 2024-08-17T02:22:55.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e40a77700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e28004810 0x7f5e3c13f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e40a77700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3c0b0980 0x7f5e3c155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e40a77700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e3c0b0df0 0x7f5e3c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e40a77700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0a83f0 con 0x7f5e3c0b0980 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e40a77700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0afb70 con 0x7f5e3c0b0df0 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e40a77700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b04f0 con 0x7f5e28004810 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e3affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3c0b0980 0x7f5e3c155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e3affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3c0b0980 0x7f5e3c155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58774/0 (socket says 172.21.15.100:58774) 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e3affd700 1 -- 172.21.15.100:0/938677904 learned_addr learned my addr 172.21.15.100:0/938677904 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e3b7fe700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e28004810 0x7f5e3c13f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.435+0000 7f5e3bfff700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e3c0b0df0 0x7f5e3c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e3affd700 1 -- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e28004810 msgr2=0x7f5e3c13f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e3affd700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e28004810 0x7f5e3c13f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e3affd700 1 -- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e3c0b0df0 msgr2=0x7f5e3c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e3affd700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e3c0b0df0 0x7f5e3c1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e3affd700 1 -- 172.21.15.100:0/938677904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0af890 con 0x7f5e3c0b0980 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e3affd700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3c0b0980 0x7f5e3c155ca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c0b0 tx=0x7f5e2401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e337fe700 1 -- 172.21.15.100:0/938677904 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24008a70 con 0x7f5e3c0b0980 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e337fe700 1 -- 172.21.15.100:0/938677904 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24001690 con 0x7f5e3c0b0980 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0afea0 con 0x7f5e3c0b0980 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e337fe700 1 -- 172.21.15.100:0/938677904 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24008380 con 0x7f5e3c0b0980 2024-08-17T02:22:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c13fd00 con 0x7f5e3c0b0980 2024-08-17T02:22:55.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c0afd00 con 0x7f5e3c0b0980 2024-08-17T02:22:55.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e337fe700 1 -- 172.21.15.100:0/938677904 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5e24008730 con 0x7f5e3c0b0980 2024-08-17T02:22:55.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e337fe700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e1c060890 0x7f5e1c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e337fe700 1 -- 172.21.15.100:0/938677904 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e24081f00 con 0x7f5e3c0b0980 2024-08-17T02:22:55.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.439+0000 7f5e3b7fe700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e1c060890 0x7f5e1c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.443+0000 7f5e3b7fe700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e1c060890 0x7f5e1c062d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c006f00 tx=0x7f5e2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.443+0000 7f5e337fe700 1 -- 172.21.15.100:0/938677904 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5e3c0afd00 con 0x7f5e3c0b0980 2024-08-17T02:22:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.603+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e3c0afd00 con 0x7f5e3c0b0980 2024-08-17T02:22:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e337fe700 1 -- 172.21.15.100:0/938677904 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f5e3c0afd00 con 0x7f5e3c0b0980 2024-08-17T02:22:55.610 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:55.610 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":33,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:22:53.933180+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:50.544046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:51.275880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:22:55.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e1c060890 msgr2=0x7f5e1c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e1c060890 0x7f5e1c062d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c006f00 tx=0x7f5e2c008040 comp rx=0 tx=0).stop 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3c0b0980 msgr2=0x7f5e3c155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3c0b0980 0x7f5e3c155ca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c0b0 tx=0x7f5e2401a000 comp rx=0 tx=0).stop 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 shutdown_connections 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5e1c060890 0x7f5e1c062d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e3c0b0df0 0x7f5e3c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3c0b0980 0x7f5e3c155ca0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 --2- 172.21.15.100:0/938677904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e28004810 0x7f5e3c13f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 >> 172.21.15.100:0/938677904 conn(0x7f5e3c0678d0 msgr2=0x7f5e3c06cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 shutdown_connections 2024-08-17T02:22:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.607+0000 7f5e40a77700 1 -- 172.21.15.100:0/938677904 wait complete. 2024-08-17T02:22:55.624 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-17T02:22:55.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:55.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b525d700 1 Processor -- start 2024-08-17T02:22:55.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b525d700 1 -- start start 2024-08-17T02:22:55.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b525d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b0d20 0x7f98b006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b525d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98b006e450 0x7f98b00b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b525d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b0141a00 con 0x7f98b00b1630 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b525d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b006e8a0 con 0x7f98b006e450 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b525d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b0140b10 con 0x7f98b00b0d20 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98af7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98b006e450 0x7f98b00b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b4a5c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f98b00b1630 0x7f98b01610f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:46698/0 (socket says 172.21.15.100:46698) 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98af7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98b006e450 0x7f98b00b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:42162/0 (socket says 172.21.15.100:42162) 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98b4a5c700 1 -- 172.21.15.100:0/1009684266 learned_addr learned my addr 172.21.15.100:0/1009684266 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:55.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98affff700 1 --2- 172.21.15.100:0/1009684266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b0d20 0x7f98b006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98af7fe700 1 -- 172.21.15.100:0/1009684266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b0d20 msgr2=0x7f98b006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98af7fe700 1 --2- 172.21.15.100:0/1009684266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b0d20 0x7f98b006df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98af7fe700 1 -- 172.21.15.100:0/1009684266 >> v1:172.21.15.123:6789/0 conn(0x7f98b00b1630 legacy=0x7f98b01610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98af7fe700 1 -- 172.21.15.100:0/1009684266 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b0140df0 con 0x7f98b006e450 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.723+0000 7f98af7fe700 1 --2- 172.21.15.100:0/1009684266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 0x7f98b00b10f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98980098e0 tx=0x7f98980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=926f1aa72ec38b05 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98aeffd700 1 -- 172.21.15.100:0/1009684266 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800b640 con 0x7f98b006e450 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98aeffd700 1 -- 172.21.15.100:0/1009684266 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989800b7a0 con 0x7f98b006e450 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98aeffd700 1 -- 172.21.15.100:0/1009684266 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800bb40 con 0x7f98b006e450 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- 172.21.15.100:0/1009684266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 msgr2=0x7f98b00b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 --2- 172.21.15.100:0/1009684266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 0x7f98b00b10f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98980098e0 tx=0x7f98980091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- 172.21.15.100:0/1009684266 shutdown_connections 2024-08-17T02:22:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 --2- 172.21.15.100:0/1009684266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 0x7f98b00b10f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 --2- 172.21.15.100:0/1009684266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b0d20 0x7f98b006df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- 172.21.15.100:0/1009684266 >> 172.21.15.100:0/1009684266 conn(0x7f98b00678d0 msgr2=0x7f98b0067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- 172.21.15.100:0/1009684266 shutdown_connections 2024-08-17T02:22:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- 172.21.15.100:0/1009684266 wait complete. 2024-08-17T02:22:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 Processor -- start 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- start start 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 0x7f98b015f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98b00b0d20 0x7f98b0159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b1630 0x7f98b0145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b015fb60 con 0x7f98b00b0d20 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b015a000 con 0x7f98b006e450 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00a7470 con 0x7f98b00b1630 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98b00b0d20 0x7f98b0159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98b00b0d20 0x7f98b0159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58780/0 (socket says 172.21.15.100:58780) 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 -- 172.21.15.100:0/1320648732 learned_addr learned my addr 172.21.15.100:0/1320648732 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98affff700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 0x7f98b015f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b4a5c700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b1630 0x7f98b0145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 -- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b1630 msgr2=0x7f98b0145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b1630 0x7f98b0145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 -- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 msgr2=0x7f98b015f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 0x7f98b015f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 -- 172.21.15.100:0/1320648732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9898009420 con 0x7f98b00b0d20 2024-08-17T02:22:55.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98affff700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 0x7f98b015f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b4a5c700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b1630 0x7f98b0145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:22:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98af7fe700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98b00b0d20 0x7f98b0159ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f989800a010 tx=0x7f9898017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98a7fff700 1 -- 172.21.15.100:0/1320648732 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989801f070 con 0x7f98b00b0d20 2024-08-17T02:22:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98a7fff700 1 -- 172.21.15.100:0/1320648732 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989801aab0 con 0x7f98b00b0d20 2024-08-17T02:22:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.727+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b00a7750 con 0x7f98b00b0d20 2024-08-17T02:22:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.731+0000 7f98a7fff700 1 -- 172.21.15.100:0/1320648732 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989801ae50 con 0x7f98b00b0d20 2024-08-17T02:22:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.731+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b00a7d10 con 0x7f98b00b0d20 2024-08-17T02:22:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.731+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b00643a0 con 0x7f98b00b0d20 2024-08-17T02:22:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.731+0000 7f98a7fff700 1 -- 172.21.15.100:0/1320648732 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9898023070 con 0x7f98b00b0d20 2024-08-17T02:22:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.731+0000 7f98a7fff700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9894060860 0x7f9894062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.731+0000 7f98a7fff700 1 -- 172.21.15.100:0/1320648732 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f989807dce0 con 0x7f98b00b0d20 2024-08-17T02:22:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.731+0000 7f98affff700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9894060860 0x7f9894062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.735+0000 7f98affff700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9894060860 0x7f9894062d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009d80 tx=0x7f98a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.743+0000 7f98a7fff700 1 -- 172.21.15.100:0/1320648732 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f989804e130 con 0x7f98b00b0d20 2024-08-17T02:22:55.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.899+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98b00b0a50 con 0x7f9894060860 2024-08-17T02:22:55.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.915+0000 7f98a7fff700 1 -- 172.21.15.100:0/1320648732 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264564 (secure 0 0 0) 0x7f98b00b0a50 con 0x7f9894060860 2024-08-17T02:22:55.924 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:55.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.919+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9894060860 msgr2=0x7f9894062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9894060860 0x7f9894062d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009d80 tx=0x7f98a0008040 comp rx=0 tx=0).stop 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98b00b0d20 msgr2=0x7f98b0159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98b00b0d20 0x7f98b0159ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f989800a010 tx=0x7f9898017000 comp rx=0 tx=0).stop 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 shutdown_connections 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9894060860 0x7f9894062d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b00b1630 0x7f98b0145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98b00b0d20 0x7f98b0159ac0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 --2- 172.21.15.100:0/1320648732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b006e450 0x7f98b015f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 >> 172.21.15.100:0/1320648732 conn(0x7f98b00678d0 msgr2=0x7f98b015bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 shutdown_connections 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:55.923+0000 7f98b525d700 1 -- 172.21.15.100:0/1320648732 wait complete. 2024-08-17T02:22:55.927 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:55.938 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-08-17T02:22:55.238320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2391076,"kb_used_data":3888,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752583644,"statfs":{"total":773094113280,"available":770645651456,"internally_reserved":0,"allocated":3981312,"data_stored":2222590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.331356"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.943974+0000","last_change":"2024-08-17T02:22:52.943974+0000","last_active":"2024-08-17T02:22:52.943974+0000","last_peered":"2024-08-17T02:22:52.943974+0000","last_clean":"2024-08-17T02:22:52.943974+0000","last_became_active":"2024-08-17T02:22:52.943395+0000","last_became_peered":"2024-08-17T02:22:52.943395+0000","last_unstale":"2024-08-17T02:22:52.943974+0000","last_undegraded":"2024-08-17T02:22:52.943974+0000","last_fullsized":"2024-08-17T02:22:52.943974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:18:06.671476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.949465+0000","last_change":"2024-08-17T02:22:52.949465+0000","last_active":"2024-08-17T02:22:52.949465+0000","last_peered":"2024-08-17T02:22:52.949465+0000","last_clean":"2024-08-17T02:22:52.949465+0000","last_became_active":"2024-08-17T02:22:52.949084+0000","last_became_peered":"2024-08-17T02:22:52.949084+0000","last_unstale":"2024-08-17T02:22:52.949465+0000","last_undegraded":"2024-08-17T02:22:52.949465+0000","last_fullsized":"2024-08-17T02:22:52.949465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:11:37.208332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.952481+0000","last_change":"2024-08-17T02:22:52.952481+0000","last_active":"2024-08-17T02:22:52.952481+0000","last_peered":"2024-08-17T02:22:52.952481+0000","last_clean":"2024-08-17T02:22:52.952481+0000","last_became_active":"2024-08-17T02:22:52.952195+0000","last_became_peered":"2024-08-17T02:22:52.952195+0000","last_unstale":"2024-08-17T02:22:52.952481+0000","last_undegraded":"2024-08-17T02:22:52.952481+0000","last_fullsized":"2024-08-17T02:22:52.952481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:55:34.680916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.950385+0000","last_change":"2024-08-17T02:22:52.950385+0000","last_active":"2024-08-17T02:22:52.950385+0000","last_peered":"2024-08-17T02:22:52.950385+0000","last_clean":"2024-08-17T02:22:52.950385+0000","last_became_active":"2024-08-17T02:22:52.950103+0000","last_became_peered":"2024-08-17T02:22:52.950103+0000","last_unstale":"2024-08-17T02:22:52.950385+0000","last_undegraded":"2024-08-17T02:22:52.950385+0000","last_fullsized":"2024-08-17T02:22:52.950385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:35:45.700871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.946427+0000","last_change":"2024-08-17T02:22:52.946427+0000","last_active":"2024-08-17T02:22:52.946427+0000","last_peered":"2024-08-17T02:22:52.946427+0000","last_clean":"2024-08-17T02:22:52.946427+0000","last_became_active":"2024-08-17T02:22:52.946175+0000","last_became_peered":"2024-08-17T02:22:52.946175+0000","last_unstale":"2024-08-17T02:22:52.946427+0000","last_undegraded":"2024-08-17T02:22:52.946427+0000","last_fullsized":"2024-08-17T02:22:52.946427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:04:16.024446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.949343+0000","last_change":"2024-08-17T02:22:52.949343+0000","last_active":"2024-08-17T02:22:52.949343+0000","last_peered":"2024-08-17T02:22:52.949343+0000","last_clean":"2024-08-17T02:22:52.949343+0000","last_became_active":"2024-08-17T02:22:52.949063+0000","last_became_peered":"2024-08-17T02:22:52.949063+0000","last_unstale":"2024-08-17T02:22:52.949343+0000","last_undegraded":"2024-08-17T02:22:52.949343+0000","last_fullsized":"2024-08-17T02:22:52.949343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:14:20.665173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953849+0000","last_change":"2024-08-17T02:22:52.953849+0000","last_active":"2024-08-17T02:22:52.953849+0000","last_peered":"2024-08-17T02:22:52.953849+0000","last_clean":"2024-08-17T02:22:52.953849+0000","last_became_active":"2024-08-17T02:22:52.953238+0000","last_became_peered":"2024-08-17T02:22:52.953238+0000","last_unstale":"2024-08-17T02:22:52.953849+0000","last_undegraded":"2024-08-17T02:22:52.953849+0000","last_fullsized":"2024-08-17T02:22:52.953849+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:50:31.084112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.955074+0000","last_change":"2024-08-17T02:22:52.955074+0000","last_active":"2024-08-17T02:22:52.955074+0000","last_peered":"2024-08-17T02:22:52.955074+0000","last_clean":"2024-08-17T02:22:52.955074+0000","last_became_active":"2024-08-17T02:22:52.954606+0000","last_became_peered":"2024-08-17T02:22:52.954606+0000","last_unstale":"2024-08-17T02:22:52.955074+0000","last_undegraded":"2024-08-17T02:22:52.955074+0000","last_fullsized":"2024-08-17T02:22:52.955074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:54:17.718209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.941513+0000","last_change":"2024-08-17T02:22:52.941513+0000","last_active":"2024-08-17T02:22:52.941513+0000","last_peered":"2024-08-17T02:22:52.941513+0000","last_clean":"2024-08-17T02:22:52.941513+0000","last_became_active":"2024-08-17T02:22:52.940959+0000","last_became_peered":"2024-08-17T02:22:52.940959+0000","last_unstale":"2024-08-17T02:22:52.941513+0000","last_undegraded":"2024-08-17T02:22:52.941513+0000","last_fullsized":"2024-08-17T02:22:52.941513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:38:21.669373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.951057+0000","last_change":"2024-08-17T02:22:52.951057+0000","last_active":"2024-08-17T02:22:52.951057+0000","last_peered":"2024-08-17T02:22:52.951057+0000","last_clean":"2024-08-17T02:22:52.951057+0000","last_became_active":"2024-08-17T02:22:52.950755+0000","last_became_peered":"2024-08-17T02:22:52.950755+0000","last_unstale":"2024-08-17T02:22:52.951057+0000","last_undegraded":"2024-08-17T02:22:52.951057+0000","last_fullsized":"2024-08-17T02:22:52.951057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:00:33.074738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953681+0000","last_change":"2024-08-17T02:22:52.953681+0000","last_active":"2024-08-17T02:22:52.953681+0000","last_peered":"2024-08-17T02:22:52.953681+0000","last_clean":"2024-08-17T02:22:52.953681+0000","last_became_active":"2024-08-17T02:22:52.952851+0000","last_became_peered":"2024-08-17T02:22:52.952851+0000","last_unstale":"2024-08-17T02:22:52.953681+0000","last_undegraded":"2024-08-17T02:22:52.953681+0000","last_fullsized":"2024-08-17T02:22:52.953681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:22:58.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.958900+0000","last_change":"2024-08-17T02:22:52.958900+0000","last_active":"2024-08-17T02:22:52.958900+0000","last_peered":"2024-08-17T02:22:52.958900+0000","last_clean":"2024-08-17T02:22:52.958900+0000","last_became_active":"2024-08-17T02:22:52.958634+0000","last_became_peered":"2024-08-17T02:22:52.958634+0000","last_unstale":"2024-08-17T02:22:52.958900+0000","last_undegraded":"2024-08-17T02:22:52.958900+0000","last_fullsized":"2024-08-17T02:22:52.958900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:42:41.132005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.944003+0000","last_change":"2024-08-17T02:22:52.944003+0000","last_active":"2024-08-17T02:22:52.944003+0000","last_peered":"2024-08-17T02:22:52.944003+0000","last_clean":"2024-08-17T02:22:52.944003+0000","last_became_active":"2024-08-17T02:22:52.943740+0000","last_became_peered":"2024-08-17T02:22:52.943740+0000","last_unstale":"2024-08-17T02:22:52.944003+0000","last_undegraded":"2024-08-17T02:22:52.944003+0000","last_fullsized":"2024-08-17T02:22:52.944003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:33.443356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945541+0000","last_change":"2024-08-17T02:22:52.945541+0000","last_active":"2024-08-17T02:22:52.945541+0000","last_peered":"2024-08-17T02:22:52.945541+0000","last_clean":"2024-08-17T02:22:52.945541+0000","last_became_active":"2024-08-17T02:22:52.945245+0000","last_became_peered":"2024-08-17T02:22:52.945245+0000","last_unstale":"2024-08-17T02:22:52.945541+0000","last_undegraded":"2024-08-17T02:22:52.945541+0000","last_fullsized":"2024-08-17T02:22:52.945541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:21:31.564013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.944148+0000","last_change":"2024-08-17T02:22:52.944148+0000","last_active":"2024-08-17T02:22:52.944148+0000","last_peered":"2024-08-17T02:22:52.944148+0000","last_clean":"2024-08-17T02:22:52.944148+0000","last_became_active":"2024-08-17T02:22:52.943844+0000","last_became_peered":"2024-08-17T02:22:52.943844+0000","last_unstale":"2024-08-17T02:22:52.944148+0000","last_undegraded":"2024-08-17T02:22:52.944148+0000","last_fullsized":"2024-08-17T02:22:52.944148+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:40:12.850376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.961941+0000","last_change":"2024-08-17T02:22:52.961941+0000","last_active":"2024-08-17T02:22:52.961941+0000","last_peered":"2024-08-17T02:22:52.961941+0000","last_clean":"2024-08-17T02:22:52.961941+0000","last_became_active":"2024-08-17T02:22:52.961699+0000","last_became_peered":"2024-08-17T02:22:52.961699+0000","last_unstale":"2024-08-17T02:22:52.961941+0000","last_undegraded":"2024-08-17T02:22:52.961941+0000","last_fullsized":"2024-08-17T02:22:52.961941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:12:53.600940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953876+0000","last_change":"2024-08-17T02:22:52.953876+0000","last_active":"2024-08-17T02:22:52.953876+0000","last_peered":"2024-08-17T02:22:52.953876+0000","last_clean":"2024-08-17T02:22:52.953876+0000","last_became_active":"2024-08-17T02:22:52.953286+0000","last_became_peered":"2024-08-17T02:22:52.953286+0000","last_unstale":"2024-08-17T02:22:52.953876+0000","last_undegraded":"2024-08-17T02:22:52.953876+0000","last_fullsized":"2024-08-17T02:22:52.953876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:55:34.680916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.955931+0000","last_change":"2024-08-17T02:22:52.955931+0000","last_active":"2024-08-17T02:22:52.955931+0000","last_peered":"2024-08-17T02:22:52.955931+0000","last_clean":"2024-08-17T02:22:52.955931+0000","last_became_active":"2024-08-17T02:22:52.955267+0000","last_became_peered":"2024-08-17T02:22:52.955267+0000","last_unstale":"2024-08-17T02:22:52.955931+0000","last_undegraded":"2024-08-17T02:22:52.955931+0000","last_fullsized":"2024-08-17T02:22:52.955931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:37:52.854917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954809+0000","last_change":"2024-08-17T02:22:52.954809+0000","last_active":"2024-08-17T02:22:52.954809+0000","last_peered":"2024-08-17T02:22:52.954809+0000","last_clean":"2024-08-17T02:22:52.954809+0000","last_became_active":"2024-08-17T02:22:52.954508+0000","last_became_peered":"2024-08-17T02:22:52.954508+0000","last_unstale":"2024-08-17T02:22:52.954809+0000","last_undegraded":"2024-08-17T02:22:52.954809+0000","last_fullsized":"2024-08-17T02:22:52.954809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:42:59.680721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.937529+0000","last_change":"2024-08-17T02:22:52.937529+0000","last_active":"2024-08-17T02:22:52.937529+0000","last_peered":"2024-08-17T02:22:52.937529+0000","last_clean":"2024-08-17T02:22:52.937529+0000","last_became_active":"2024-08-17T02:22:52.937209+0000","last_became_peered":"2024-08-17T02:22:52.937209+0000","last_unstale":"2024-08-17T02:22:52.937529+0000","last_undegraded":"2024-08-17T02:22:52.937529+0000","last_fullsized":"2024-08-17T02:22:52.937529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:55:56.713536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.940932+0000","last_change":"2024-08-17T02:22:52.940932+0000","last_active":"2024-08-17T02:22:52.940932+0000","last_peered":"2024-08-17T02:22:52.940932+0000","last_clean":"2024-08-17T02:22:52.940932+0000","last_became_active":"2024-08-17T02:22:52.940511+0000","last_became_peered":"2024-08-17T02:22:52.940511+0000","last_unstale":"2024-08-17T02:22:52.940932+0000","last_undegraded":"2024-08-17T02:22:52.940932+0000","last_fullsized":"2024-08-17T02:22:52.940932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:11.073301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.949598+0000","last_change":"2024-08-17T02:22:52.949598+0000","last_active":"2024-08-17T02:22:52.949598+0000","last_peered":"2024-08-17T02:22:52.949598+0000","last_clean":"2024-08-17T02:22:52.949598+0000","last_became_active":"2024-08-17T02:22:52.949310+0000","last_became_peered":"2024-08-17T02:22:52.949310+0000","last_unstale":"2024-08-17T02:22:52.949598+0000","last_undegraded":"2024-08-17T02:22:52.949598+0000","last_fullsized":"2024-08-17T02:22:52.949598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:21:51.352920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.946416+0000","last_change":"2024-08-17T02:22:52.946416+0000","last_active":"2024-08-17T02:22:52.946416+0000","last_peered":"2024-08-17T02:22:52.946416+0000","last_clean":"2024-08-17T02:22:52.946416+0000","last_became_active":"2024-08-17T02:22:52.946096+0000","last_became_peered":"2024-08-17T02:22:52.946096+0000","last_unstale":"2024-08-17T02:22:52.946416+0000","last_undegraded":"2024-08-17T02:22:52.946416+0000","last_fullsized":"2024-08-17T02:22:52.946416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:58:05.009785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.936651+0000","last_change":"2024-08-17T02:22:52.936651+0000","last_active":"2024-08-17T02:22:52.936651+0000","last_peered":"2024-08-17T02:22:52.936651+0000","last_clean":"2024-08-17T02:22:52.936651+0000","last_became_active":"2024-08-17T02:22:52.936235+0000","last_became_peered":"2024-08-17T02:22:52.936235+0000","last_unstale":"2024-08-17T02:22:52.936651+0000","last_undegraded":"2024-08-17T02:22:52.936651+0000","last_fullsized":"2024-08-17T02:22:52.936651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:52:25.176345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.956048+0000","last_change":"2024-08-17T02:22:52.956048+0000","last_active":"2024-08-17T02:22:52.956048+0000","last_peered":"2024-08-17T02:22:52.956048+0000","last_clean":"2024-08-17T02:22:52.956048+0000","last_became_active":"2024-08-17T02:22:52.955575+0000","last_became_peered":"2024-08-17T02:22:52.955575+0000","last_unstale":"2024-08-17T02:22:52.956048+0000","last_undegraded":"2024-08-17T02:22:52.956048+0000","last_fullsized":"2024-08-17T02:22:52.956048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:21:51.352920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.956101+0000","last_change":"2024-08-17T02:22:52.956101+0000","last_active":"2024-08-17T02:22:52.956101+0000","last_peered":"2024-08-17T02:22:52.956101+0000","last_clean":"2024-08-17T02:22:52.956101+0000","last_became_active":"2024-08-17T02:22:52.955609+0000","last_became_peered":"2024-08-17T02:22:52.955609+0000","last_unstale":"2024-08-17T02:22:52.956101+0000","last_undegraded":"2024-08-17T02:22:52.956101+0000","last_fullsized":"2024-08-17T02:22:52.956101+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:41:45.005621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.951193+0000","last_change":"2024-08-17T02:22:52.951193+0000","last_active":"2024-08-17T02:22:52.951193+0000","last_peered":"2024-08-17T02:22:52.951193+0000","last_clean":"2024-08-17T02:22:52.951193+0000","last_became_active":"2024-08-17T02:22:52.950571+0000","last_became_peered":"2024-08-17T02:22:52.950571+0000","last_unstale":"2024-08-17T02:22:52.951193+0000","last_undegraded":"2024-08-17T02:22:52.951193+0000","last_fullsized":"2024-08-17T02:22:52.951193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:26:14.783070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.941997+0000","last_change":"2024-08-17T02:22:52.941997+0000","last_active":"2024-08-17T02:22:52.941997+0000","last_peered":"2024-08-17T02:22:52.941997+0000","last_clean":"2024-08-17T02:22:52.941997+0000","last_became_active":"2024-08-17T02:22:52.941710+0000","last_became_peered":"2024-08-17T02:22:52.941710+0000","last_unstale":"2024-08-17T02:22:52.941997+0000","last_undegraded":"2024-08-17T02:22:52.941997+0000","last_fullsized":"2024-08-17T02:22:52.941997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:50:31.084112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945964+0000","last_change":"2024-08-17T02:22:52.945964+0000","last_active":"2024-08-17T02:22:52.945964+0000","last_peered":"2024-08-17T02:22:52.945964+0000","last_clean":"2024-08-17T02:22:52.945964+0000","last_became_active":"2024-08-17T02:22:52.945625+0000","last_became_peered":"2024-08-17T02:22:52.945625+0000","last_unstale":"2024-08-17T02:22:52.945964+0000","last_undegraded":"2024-08-17T02:22:52.945964+0000","last_fullsized":"2024-08-17T02:22:52.945964+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:35:50.869077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953625+0000","last_change":"2024-08-17T02:22:52.953625+0000","last_active":"2024-08-17T02:22:52.953625+0000","last_peered":"2024-08-17T02:22:52.953625+0000","last_clean":"2024-08-17T02:22:52.953625+0000","last_became_active":"2024-08-17T02:22:52.952707+0000","last_became_peered":"2024-08-17T02:22:52.952707+0000","last_unstale":"2024-08-17T02:22:52.953625+0000","last_undegraded":"2024-08-17T02:22:52.953625+0000","last_fullsized":"2024-08-17T02:22:52.953625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:42:41.132005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.947674+0000","last_change":"2024-08-17T02:22:52.947674+0000","last_active":"2024-08-17T02:22:52.947674+0000","last_peered":"2024-08-17T02:22:52.947674+0000","last_clean":"2024-08-17T02:22:52.947674+0000","last_became_active":"2024-08-17T02:22:52.943447+0000","last_became_peered":"2024-08-17T02:22:52.943447+0000","last_unstale":"2024-08-17T02:22:52.947674+0000","last_undegraded":"2024-08-17T02:22:52.947674+0000","last_fullsized":"2024-08-17T02:22:52.947674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:42:19.890202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953903+0000","last_change":"2024-08-17T02:22:52.953903+0000","last_active":"2024-08-17T02:22:52.953903+0000","last_peered":"2024-08-17T02:22:52.953903+0000","last_clean":"2024-08-17T02:22:52.953903+0000","last_became_active":"2024-08-17T02:22:52.953196+0000","last_became_peered":"2024-08-17T02:22:52.953196+0000","last_unstale":"2024-08-17T02:22:52.953903+0000","last_undegraded":"2024-08-17T02:22:52.953903+0000","last_fullsized":"2024-08-17T02:22:52.953903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:08:05.776194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.947018+0000","last_change":"2024-08-17T02:22:52.947018+0000","last_active":"2024-08-17T02:22:52.947018+0000","last_peered":"2024-08-17T02:22:52.947018+0000","last_clean":"2024-08-17T02:22:52.947018+0000","last_became_active":"2024-08-17T02:22:52.946756+0000","last_became_peered":"2024-08-17T02:22:52.946756+0000","last_unstale":"2024-08-17T02:22:52.947018+0000","last_undegraded":"2024-08-17T02:22:52.947018+0000","last_fullsized":"2024-08-17T02:22:52.947018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:26:19.802145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.948296+0000","last_change":"2024-08-17T02:22:51.939680+0000","last_active":"2024-08-17T02:22:52.948296+0000","last_peered":"2024-08-17T02:22:52.948296+0000","last_clean":"2024-08-17T02:22:52.948296+0000","last_became_active":"2024-08-17T02:22:51.939012+0000","last_became_peered":"2024-08-17T02:22:51.939012+0000","last_unstale":"2024-08-17T02:22:52.948296+0000","last_undegraded":"2024-08-17T02:22:52.948296+0000","last_fullsized":"2024-08-17T02:22:52.948296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:56:48.264910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.948379+0000","last_change":"2024-08-17T02:22:52.948379+0000","last_active":"2024-08-17T02:22:52.948379+0000","last_peered":"2024-08-17T02:22:52.948379+0000","last_clean":"2024-08-17T02:22:52.948379+0000","last_became_active":"2024-08-17T02:22:52.948103+0000","last_became_peered":"2024-08-17T02:22:52.948103+0000","last_unstale":"2024-08-17T02:22:52.948379+0000","last_undegraded":"2024-08-17T02:22:52.948379+0000","last_fullsized":"2024-08-17T02:22:52.948379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:35:50.869077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.926862+0000","last_change":"2024-08-17T02:22:51.949277+0000","last_active":"2024-08-17T02:22:52.926862+0000","last_peered":"2024-08-17T02:22:52.926862+0000","last_clean":"2024-08-17T02:22:52.926862+0000","last_became_active":"2024-08-17T02:22:51.948908+0000","last_became_peered":"2024-08-17T02:22:51.948908+0000","last_unstale":"2024-08-17T02:22:52.926862+0000","last_undegraded":"2024-08-17T02:22:52.926862+0000","last_fullsized":"2024-08-17T02:22:52.926862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:21:48.264201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.959016+0000","last_change":"2024-08-17T02:22:52.959016+0000","last_active":"2024-08-17T02:22:52.959016+0000","last_peered":"2024-08-17T02:22:52.959016+0000","last_clean":"2024-08-17T02:22:52.959016+0000","last_became_active":"2024-08-17T02:22:52.958715+0000","last_became_peered":"2024-08-17T02:22:52.958715+0000","last_unstale":"2024-08-17T02:22:52.959016+0000","last_undegraded":"2024-08-17T02:22:52.959016+0000","last_fullsized":"2024-08-17T02:22:52.959016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:39:13.398435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.958069+0000","last_change":"2024-08-17T02:22:51.945075+0000","last_active":"2024-08-17T02:22:52.958069+0000","last_peered":"2024-08-17T02:22:52.958069+0000","last_clean":"2024-08-17T02:22:52.958069+0000","last_became_active":"2024-08-17T02:22:51.944843+0000","last_became_peered":"2024-08-17T02:22:51.944843+0000","last_unstale":"2024-08-17T02:22:52.958069+0000","last_undegraded":"2024-08-17T02:22:52.958069+0000","last_fullsized":"2024-08-17T02:22:52.958069+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:44:45.251715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.951387+0000","last_change":"2024-08-17T02:22:52.951387+0000","last_active":"2024-08-17T02:22:52.951387+0000","last_peered":"2024-08-17T02:22:52.951387+0000","last_clean":"2024-08-17T02:22:52.951387+0000","last_became_active":"2024-08-17T02:22:52.950936+0000","last_became_peered":"2024-08-17T02:22:52.950936+0000","last_unstale":"2024-08-17T02:22:52.951387+0000","last_undegraded":"2024-08-17T02:22:52.951387+0000","last_fullsized":"2024-08-17T02:22:52.951387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:31.460732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.957448+0000","last_change":"2024-08-17T02:22:51.944451+0000","last_active":"2024-08-17T02:22:52.957448+0000","last_peered":"2024-08-17T02:22:52.957448+0000","last_clean":"2024-08-17T02:22:52.957448+0000","last_became_active":"2024-08-17T02:22:51.944175+0000","last_became_peered":"2024-08-17T02:22:51.944175+0000","last_unstale":"2024-08-17T02:22:52.957448+0000","last_undegraded":"2024-08-17T02:22:52.957448+0000","last_fullsized":"2024-08-17T02:22:52.957448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:51:36.515524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953946+0000","last_change":"2024-08-17T02:22:52.953946+0000","last_active":"2024-08-17T02:22:52.953946+0000","last_peered":"2024-08-17T02:22:52.953946+0000","last_clean":"2024-08-17T02:22:52.953946+0000","last_became_active":"2024-08-17T02:22:52.953011+0000","last_became_peered":"2024-08-17T02:22:52.953011+0000","last_unstale":"2024-08-17T02:22:52.953946+0000","last_undegraded":"2024-08-17T02:22:52.953946+0000","last_fullsized":"2024-08-17T02:22:52.953946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:52:25.176345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.930033+0000","last_change":"2024-08-17T02:22:51.935261+0000","last_active":"2024-08-17T02:22:52.930033+0000","last_peered":"2024-08-17T02:22:52.930033+0000","last_clean":"2024-08-17T02:22:52.930033+0000","last_became_active":"2024-08-17T02:22:51.934981+0000","last_became_peered":"2024-08-17T02:22:51.934981+0000","last_unstale":"2024-08-17T02:22:52.930033+0000","last_undegraded":"2024-08-17T02:22:52.930033+0000","last_fullsized":"2024-08-17T02:22:52.930033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:03:15.711105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.942836+0000","last_change":"2024-08-17T02:22:52.942836+0000","last_active":"2024-08-17T02:22:52.942836+0000","last_peered":"2024-08-17T02:22:52.942836+0000","last_clean":"2024-08-17T02:22:52.942836+0000","last_became_active":"2024-08-17T02:22:52.941938+0000","last_became_peered":"2024-08-17T02:22:52.941938+0000","last_unstale":"2024-08-17T02:22:52.942836+0000","last_undegraded":"2024-08-17T02:22:52.942836+0000","last_fullsized":"2024-08-17T02:22:52.942836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:54:17.718209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.926530+0000","last_change":"2024-08-17T02:22:51.937290+0000","last_active":"2024-08-17T02:22:52.926530+0000","last_peered":"2024-08-17T02:22:52.926530+0000","last_clean":"2024-08-17T02:22:52.926530+0000","last_became_active":"2024-08-17T02:22:51.937023+0000","last_became_peered":"2024-08-17T02:22:51.937023+0000","last_unstale":"2024-08-17T02:22:52.926530+0000","last_undegraded":"2024-08-17T02:22:52.926530+0000","last_fullsized":"2024-08-17T02:22:52.926530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:45:01.390768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945299+0000","last_change":"2024-08-17T02:22:52.945299+0000","last_active":"2024-08-17T02:22:52.945299+0000","last_peered":"2024-08-17T02:22:52.945299+0000","last_clean":"2024-08-17T02:22:52.945299+0000","last_became_active":"2024-08-17T02:22:52.944972+0000","last_became_peered":"2024-08-17T02:22:52.944972+0000","last_unstale":"2024-08-17T02:22:52.945299+0000","last_undegraded":"2024-08-17T02:22:52.945299+0000","last_fullsized":"2024-08-17T02:22:52.945299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:44:46.255701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.944590+0000","last_change":"2024-08-17T02:22:51.941438+0000","last_active":"2024-08-17T02:22:51.944590+0000","last_peered":"2024-08-17T02:22:51.944590+0000","last_clean":"2024-08-17T02:22:51.944590+0000","last_became_active":"2024-08-17T02:22:51.941115+0000","last_became_peered":"2024-08-17T02:22:51.941115+0000","last_unstale":"2024-08-17T02:22:51.944590+0000","last_undegraded":"2024-08-17T02:22:51.944590+0000","last_fullsized":"2024-08-17T02:22:51.944590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:07:14.226381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.947951+0000","last_change":"2024-08-17T02:22:52.947951+0000","last_active":"2024-08-17T02:22:52.947951+0000","last_peered":"2024-08-17T02:22:52.947951+0000","last_clean":"2024-08-17T02:22:52.947951+0000","last_became_active":"2024-08-17T02:22:52.947619+0000","last_became_peered":"2024-08-17T02:22:52.947619+0000","last_unstale":"2024-08-17T02:22:52.947951+0000","last_undegraded":"2024-08-17T02:22:52.947951+0000","last_fullsized":"2024-08-17T02:22:52.947951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:11.073301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.937553+0000","last_change":"2024-08-17T02:22:51.934206+0000","last_active":"2024-08-17T02:22:51.937553+0000","last_peered":"2024-08-17T02:22:51.937553+0000","last_clean":"2024-08-17T02:22:51.937553+0000","last_became_active":"2024-08-17T02:22:51.933897+0000","last_became_peered":"2024-08-17T02:22:51.933897+0000","last_unstale":"2024-08-17T02:22:51.937553+0000","last_undegraded":"2024-08-17T02:22:51.937553+0000","last_fullsized":"2024-08-17T02:22:51.937553+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:30:53.019442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.949404+0000","last_change":"2024-08-17T02:22:52.949404+0000","last_active":"2024-08-17T02:22:52.949404+0000","last_peered":"2024-08-17T02:22:52.949404+0000","last_clean":"2024-08-17T02:22:52.949404+0000","last_became_active":"2024-08-17T02:22:52.949163+0000","last_became_peered":"2024-08-17T02:22:52.949163+0000","last_unstale":"2024-08-17T02:22:52.949404+0000","last_undegraded":"2024-08-17T02:22:52.949404+0000","last_fullsized":"2024-08-17T02:22:52.949404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:23:35.890471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.943556+0000","last_change":"2024-08-17T02:22:51.938623+0000","last_active":"2024-08-17T02:22:51.943556+0000","last_peered":"2024-08-17T02:22:51.943556+0000","last_clean":"2024-08-17T02:22:51.943556+0000","last_became_active":"2024-08-17T02:22:51.938275+0000","last_became_peered":"2024-08-17T02:22:51.938275+0000","last_unstale":"2024-08-17T02:22:51.943556+0000","last_undegraded":"2024-08-17T02:22:51.943556+0000","last_fullsized":"2024-08-17T02:22:51.943556+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:35:49.865091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954698+0000","last_change":"2024-08-17T02:22:52.954698+0000","last_active":"2024-08-17T02:22:52.954698+0000","last_peered":"2024-08-17T02:22:52.954698+0000","last_clean":"2024-08-17T02:22:52.954698+0000","last_became_active":"2024-08-17T02:22:52.954361+0000","last_became_peered":"2024-08-17T02:22:52.954361+0000","last_unstale":"2024-08-17T02:22:52.954698+0000","last_undegraded":"2024-08-17T02:22:52.954698+0000","last_fullsized":"2024-08-17T02:22:52.954698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:18:13.634439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.950267+0000","last_change":"2024-08-17T02:22:51.949130+0000","last_active":"2024-08-17T02:22:51.950267+0000","last_peered":"2024-08-17T02:22:51.950267+0000","last_clean":"2024-08-17T02:22:51.950267+0000","last_became_active":"2024-08-17T02:22:51.948561+0000","last_became_peered":"2024-08-17T02:22:51.948561+0000","last_unstale":"2024-08-17T02:22:51.950267+0000","last_undegraded":"2024-08-17T02:22:51.950267+0000","last_fullsized":"2024-08-17T02:22:51.950267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:32.439370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.959112+0000","last_change":"2024-08-17T02:22:52.959112+0000","last_active":"2024-08-17T02:22:52.959112+0000","last_peered":"2024-08-17T02:22:52.959112+0000","last_clean":"2024-08-17T02:22:52.959112+0000","last_became_active":"2024-08-17T02:22:52.958781+0000","last_became_peered":"2024-08-17T02:22:52.958781+0000","last_unstale":"2024-08-17T02:22:52.959112+0000","last_undegraded":"2024-08-17T02:22:52.959112+0000","last_fullsized":"2024-08-17T02:22:52.959112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:22:22.781465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.955009+0000","last_change":"2024-08-17T02:22:51.936517+0000","last_active":"2024-08-17T02:22:52.955009+0000","last_peered":"2024-08-17T02:22:52.955009+0000","last_clean":"2024-08-17T02:22:52.955009+0000","last_became_active":"2024-08-17T02:22:51.936066+0000","last_became_peered":"2024-08-17T02:22:51.936066+0000","last_unstale":"2024-08-17T02:22:52.955009+0000","last_undegraded":"2024-08-17T02:22:52.955009+0000","last_fullsized":"2024-08-17T02:22:52.955009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:38:02.744666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.952621+0000","last_change":"2024-08-17T02:22:52.952621+0000","last_active":"2024-08-17T02:22:52.952621+0000","last_peered":"2024-08-17T02:22:52.952621+0000","last_clean":"2024-08-17T02:22:52.952621+0000","last_became_active":"2024-08-17T02:22:52.952328+0000","last_became_peered":"2024-08-17T02:22:52.952328+0000","last_unstale":"2024-08-17T02:22:52.952621+0000","last_undegraded":"2024-08-17T02:22:52.952621+0000","last_fullsized":"2024-08-17T02:22:52.952621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:12:53.600940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.962292+0000","last_change":"2024-08-17T02:22:51.950505+0000","last_active":"2024-08-17T02:22:52.962292+0000","last_peered":"2024-08-17T02:22:52.962292+0000","last_clean":"2024-08-17T02:22:52.962292+0000","last_became_active":"2024-08-17T02:22:51.950210+0000","last_became_peered":"2024-08-17T02:22:51.950210+0000","last_unstale":"2024-08-17T02:22:52.962292+0000","last_undegraded":"2024-08-17T02:22:52.962292+0000","last_fullsized":"2024-08-17T02:22:52.962292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:47:16.297181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.950187+0000","last_change":"2024-08-17T02:22:52.950187+0000","last_active":"2024-08-17T02:22:52.950187+0000","last_peered":"2024-08-17T02:22:52.950187+0000","last_clean":"2024-08-17T02:22:52.950187+0000","last_became_active":"2024-08-17T02:22:52.949903+0000","last_became_peered":"2024-08-17T02:22:52.949903+0000","last_unstale":"2024-08-17T02:22:52.950187+0000","last_undegraded":"2024-08-17T02:22:52.950187+0000","last_fullsized":"2024-08-17T02:22:52.950187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:38:21.669373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945963+0000","last_change":"2024-08-17T02:22:51.935474+0000","last_active":"2024-08-17T02:22:52.945963+0000","last_peered":"2024-08-17T02:22:52.945963+0000","last_clean":"2024-08-17T02:22:52.945963+0000","last_became_active":"2024-08-17T02:22:51.934609+0000","last_became_peered":"2024-08-17T02:22:51.934609+0000","last_unstale":"2024-08-17T02:22:52.945963+0000","last_undegraded":"2024-08-17T02:22:52.945963+0000","last_fullsized":"2024-08-17T02:22:52.945963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:38:07.034881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945501+0000","last_change":"2024-08-17T02:22:52.945501+0000","last_active":"2024-08-17T02:22:52.945501+0000","last_peered":"2024-08-17T02:22:52.945501+0000","last_clean":"2024-08-17T02:22:52.945501+0000","last_became_active":"2024-08-17T02:22:52.944947+0000","last_became_peered":"2024-08-17T02:22:52.944947+0000","last_unstale":"2024-08-17T02:22:52.945501+0000","last_undegraded":"2024-08-17T02:22:52.945501+0000","last_fullsized":"2024-08-17T02:22:52.945501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:03:55.493198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.945899+0000","last_change":"2024-08-17T02:22:51.939608+0000","last_active":"2024-08-17T02:22:51.945899+0000","last_peered":"2024-08-17T02:22:51.945899+0000","last_clean":"2024-08-17T02:22:51.945899+0000","last_became_active":"2024-08-17T02:22:51.939266+0000","last_became_peered":"2024-08-17T02:22:51.939266+0000","last_unstale":"2024-08-17T02:22:51.945899+0000","last_undegraded":"2024-08-17T02:22:51.945899+0000","last_fullsized":"2024-08-17T02:22:51.945899+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:07:16.906696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954900+0000","last_change":"2024-08-17T02:22:52.954900+0000","last_active":"2024-08-17T02:22:52.954900+0000","last_peered":"2024-08-17T02:22:52.954900+0000","last_clean":"2024-08-17T02:22:52.954900+0000","last_became_active":"2024-08-17T02:22:52.954225+0000","last_became_peered":"2024-08-17T02:22:52.954225+0000","last_unstale":"2024-08-17T02:22:52.954900+0000","last_undegraded":"2024-08-17T02:22:52.954900+0000","last_fullsized":"2024-08-17T02:22:52.954900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:40:12.850376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.944569+0000","last_change":"2024-08-17T02:22:51.938614+0000","last_active":"2024-08-17T02:22:51.944569+0000","last_peered":"2024-08-17T02:22:51.944569+0000","last_clean":"2024-08-17T02:22:51.944569+0000","last_became_active":"2024-08-17T02:22:51.938329+0000","last_became_peered":"2024-08-17T02:22:51.938329+0000","last_unstale":"2024-08-17T02:22:51.944569+0000","last_undegraded":"2024-08-17T02:22:51.944569+0000","last_fullsized":"2024-08-17T02:22:51.944569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:18:05.667490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.950581+0000","last_change":"2024-08-17T02:22:52.950581+0000","last_active":"2024-08-17T02:22:52.950581+0000","last_peered":"2024-08-17T02:22:52.950581+0000","last_clean":"2024-08-17T02:22:52.950581+0000","last_became_active":"2024-08-17T02:22:52.950290+0000","last_became_peered":"2024-08-17T02:22:52.950290+0000","last_unstale":"2024-08-17T02:22:52.950581+0000","last_undegraded":"2024-08-17T02:22:52.950581+0000","last_fullsized":"2024-08-17T02:22:52.950581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:44:46.255701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.961980+0000","last_change":"2024-08-17T02:22:51.945073+0000","last_active":"2024-08-17T02:22:52.961980+0000","last_peered":"2024-08-17T02:22:52.961980+0000","last_clean":"2024-08-17T02:22:52.961980+0000","last_became_active":"2024-08-17T02:22:51.944804+0000","last_became_peered":"2024-08-17T02:22:51.944804+0000","last_unstale":"2024-08-17T02:22:52.961980+0000","last_undegraded":"2024-08-17T02:22:52.961980+0000","last_fullsized":"2024-08-17T02:22:52.961980+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:22.299573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.939950+0000","last_change":"2024-08-17T02:22:52.939950+0000","last_active":"2024-08-17T02:22:52.939950+0000","last_peered":"2024-08-17T02:22:52.939950+0000","last_clean":"2024-08-17T02:22:52.939950+0000","last_became_active":"2024-08-17T02:22:52.939166+0000","last_became_peered":"2024-08-17T02:22:52.939166+0000","last_unstale":"2024-08-17T02:22:52.939950+0000","last_undegraded":"2024-08-17T02:22:52.939950+0000","last_fullsized":"2024-08-17T02:22:52.939950+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:07:17.670198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.938978+0000","last_change":"2024-08-17T02:22:51.945974+0000","last_active":"2024-08-17T02:22:52.938978+0000","last_peered":"2024-08-17T02:22:52.938978+0000","last_clean":"2024-08-17T02:22:52.938978+0000","last_became_active":"2024-08-17T02:22:51.945291+0000","last_became_peered":"2024-08-17T02:22:51.945291+0000","last_unstale":"2024-08-17T02:22:52.938978+0000","last_undegraded":"2024-08-17T02:22:52.938978+0000","last_fullsized":"2024-08-17T02:22:52.938978+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:47:14.245060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.933656+0000","last_change":"2024-08-17T02:22:52.933656+0000","last_active":"2024-08-17T02:22:52.933656+0000","last_peered":"2024-08-17T02:22:52.933656+0000","last_clean":"2024-08-17T02:22:52.933656+0000","last_became_active":"2024-08-17T02:22:52.933236+0000","last_became_peered":"2024-08-17T02:22:52.933236+0000","last_unstale":"2024-08-17T02:22:52.933656+0000","last_undegraded":"2024-08-17T02:22:52.933656+0000","last_fullsized":"2024-08-17T02:22:52.933656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:22:58.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.946432+0000","last_change":"2024-08-17T02:22:51.933910+0000","last_active":"2024-08-17T02:22:52.946432+0000","last_peered":"2024-08-17T02:22:52.946432+0000","last_clean":"2024-08-17T02:22:52.946432+0000","last_became_active":"2024-08-17T02:22:51.933605+0000","last_became_peered":"2024-08-17T02:22:51.933605+0000","last_unstale":"2024-08-17T02:22:52.946432+0000","last_undegraded":"2024-08-17T02:22:52.946432+0000","last_fullsized":"2024-08-17T02:22:52.946432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:50.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.952363+0000","last_change":"2024-08-17T02:22:52.952363+0000","last_active":"2024-08-17T02:22:52.952363+0000","last_peered":"2024-08-17T02:22:52.952363+0000","last_clean":"2024-08-17T02:22:52.952363+0000","last_became_active":"2024-08-17T02:22:52.951977+0000","last_became_peered":"2024-08-17T02:22:52.951977+0000","last_unstale":"2024-08-17T02:22:52.952363+0000","last_undegraded":"2024-08-17T02:22:52.952363+0000","last_fullsized":"2024-08-17T02:22:52.952363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:07:15.230367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.929534+0000","last_change":"2024-08-17T02:22:51.938548+0000","last_active":"2024-08-17T02:22:52.929534+0000","last_peered":"2024-08-17T02:22:52.929534+0000","last_clean":"2024-08-17T02:22:52.929534+0000","last_became_active":"2024-08-17T02:22:51.938244+0000","last_became_peered":"2024-08-17T02:22:51.938244+0000","last_unstale":"2024-08-17T02:22:52.929534+0000","last_undegraded":"2024-08-17T02:22:52.929534+0000","last_fullsized":"2024-08-17T02:22:52.929534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:13:19.985389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954074+0000","last_change":"2024-08-17T02:22:52.954074+0000","last_active":"2024-08-17T02:22:52.954074+0000","last_peered":"2024-08-17T02:22:52.954074+0000","last_clean":"2024-08-17T02:22:52.954074+0000","last_became_active":"2024-08-17T02:22:52.953748+0000","last_became_peered":"2024-08-17T02:22:52.953748+0000","last_unstale":"2024-08-17T02:22:52.954074+0000","last_undegraded":"2024-08-17T02:22:52.954074+0000","last_fullsized":"2024-08-17T02:22:52.954074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:12.757615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.946965+0000","last_change":"2024-08-17T02:22:51.945137+0000","last_active":"2024-08-17T02:22:51.946965+0000","last_peered":"2024-08-17T02:22:51.946965+0000","last_clean":"2024-08-17T02:22:51.946965+0000","last_became_active":"2024-08-17T02:22:51.944710+0000","last_became_peered":"2024-08-17T02:22:51.944710+0000","last_unstale":"2024-08-17T02:22:51.946965+0000","last_undegraded":"2024-08-17T02:22:51.946965+0000","last_fullsized":"2024-08-17T02:22:51.946965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:56:48.264910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.940442+0000","last_change":"2024-08-17T02:22:52.940442+0000","last_active":"2024-08-17T02:22:52.940442+0000","last_peered":"2024-08-17T02:22:52.940442+0000","last_clean":"2024-08-17T02:22:52.940442+0000","last_became_active":"2024-08-17T02:22:52.940137+0000","last_became_peered":"2024-08-17T02:22:52.940137+0000","last_unstale":"2024-08-17T02:22:52.940442+0000","last_undegraded":"2024-08-17T02:22:52.940442+0000","last_fullsized":"2024-08-17T02:22:52.940442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:09:58.358246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.952840+0000","last_change":"2024-08-17T02:22:51.948906+0000","last_active":"2024-08-17T02:22:52.952840+0000","last_peered":"2024-08-17T02:22:52.952840+0000","last_clean":"2024-08-17T02:22:52.952840+0000","last_became_active":"2024-08-17T02:22:51.948578+0000","last_became_peered":"2024-08-17T02:22:51.948578+0000","last_unstale":"2024-08-17T02:22:52.952840+0000","last_undegraded":"2024-08-17T02:22:52.952840+0000","last_fullsized":"2024-08-17T02:22:52.952840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:11.294023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.943423+0000","last_change":"2024-08-17T02:22:52.943423+0000","last_active":"2024-08-17T02:22:52.943423+0000","last_peered":"2024-08-17T02:22:52.943423+0000","last_clean":"2024-08-17T02:22:52.943423+0000","last_became_active":"2024-08-17T02:22:52.941153+0000","last_became_peered":"2024-08-17T02:22:52.941153+0000","last_unstale":"2024-08-17T02:22:52.943423+0000","last_undegraded":"2024-08-17T02:22:52.943423+0000","last_fullsized":"2024-08-17T02:22:52.943423+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:12.298009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.949666+0000","last_change":"2024-08-17T02:22:51.946026+0000","last_active":"2024-08-17T02:22:51.949666+0000","last_peered":"2024-08-17T02:22:51.949666+0000","last_clean":"2024-08-17T02:22:51.949666+0000","last_became_active":"2024-08-17T02:22:51.945731+0000","last_became_peered":"2024-08-17T02:22:51.945731+0000","last_unstale":"2024-08-17T02:22:51.949666+0000","last_undegraded":"2024-08-17T02:22:51.949666+0000","last_fullsized":"2024-08-17T02:22:51.949666+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:16:14.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945640+0000","last_change":"2024-08-17T02:22:52.945640+0000","last_active":"2024-08-17T02:22:52.945640+0000","last_peered":"2024-08-17T02:22:52.945640+0000","last_clean":"2024-08-17T02:22:52.945640+0000","last_became_active":"2024-08-17T02:22:52.945334+0000","last_became_peered":"2024-08-17T02:22:52.945334+0000","last_unstale":"2024-08-17T02:22:52.945640+0000","last_undegraded":"2024-08-17T02:22:52.945640+0000","last_fullsized":"2024-08-17T02:22:52.945640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:21:49.268187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.944942+0000","last_change":"2024-08-17T02:22:51.943706+0000","last_active":"2024-08-17T02:22:51.944942+0000","last_peered":"2024-08-17T02:22:51.944942+0000","last_clean":"2024-08-17T02:22:51.944942+0000","last_became_active":"2024-08-17T02:22:51.943417+0000","last_became_peered":"2024-08-17T02:22:51.943417+0000","last_unstale":"2024-08-17T02:22:51.944942+0000","last_undegraded":"2024-08-17T02:22:51.944942+0000","last_fullsized":"2024-08-17T02:22:51.944942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:43:04.344779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954567+0000","last_change":"2024-08-17T02:22:52.954567+0000","last_active":"2024-08-17T02:22:52.954567+0000","last_peered":"2024-08-17T02:22:52.954567+0000","last_clean":"2024-08-17T02:22:52.954567+0000","last_became_active":"2024-08-17T02:22:52.954039+0000","last_became_peered":"2024-08-17T02:22:52.954039+0000","last_unstale":"2024-08-17T02:22:52.954567+0000","last_undegraded":"2024-08-17T02:22:52.954567+0000","last_fullsized":"2024-08-17T02:22:52.954567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:38.069670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.941434+0000","last_change":"2024-08-17T02:22:51.939127+0000","last_active":"2024-08-17T02:22:51.941434+0000","last_peered":"2024-08-17T02:22:51.941434+0000","last_clean":"2024-08-17T02:22:51.941434+0000","last_became_active":"2024-08-17T02:22:51.938849+0000","last_became_peered":"2024-08-17T02:22:51.938849+0000","last_unstale":"2024-08-17T02:22:51.941434+0000","last_undegraded":"2024-08-17T02:22:51.941434+0000","last_fullsized":"2024-08-17T02:22:51.941434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:58:04.005799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.938331+0000","last_change":"2024-08-17T02:22:52.938331+0000","last_active":"2024-08-17T02:22:52.938331+0000","last_peered":"2024-08-17T02:22:52.938331+0000","last_clean":"2024-08-17T02:22:52.938331+0000","last_became_active":"2024-08-17T02:22:52.938010+0000","last_became_peered":"2024-08-17T02:22:52.938010+0000","last_unstale":"2024-08-17T02:22:52.938331+0000","last_undegraded":"2024-08-17T02:22:52.938331+0000","last_fullsized":"2024-08-17T02:22:52.938331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:50:24.419251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.958386+0000","last_change":"2024-08-17T02:22:51.946920+0000","last_active":"2024-08-17T02:22:52.958386+0000","last_peered":"2024-08-17T02:22:52.958386+0000","last_clean":"2024-08-17T02:22:52.958386+0000","last_became_active":"2024-08-17T02:22:51.946636+0000","last_became_peered":"2024-08-17T02:22:51.946636+0000","last_unstale":"2024-08-17T02:22:52.958386+0000","last_undegraded":"2024-08-17T02:22:52.958386+0000","last_fullsized":"2024-08-17T02:22:52.958386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:16:14.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.941627+0000","last_change":"2024-08-17T02:22:52.941627+0000","last_active":"2024-08-17T02:22:52.941627+0000","last_peered":"2024-08-17T02:22:52.941627+0000","last_clean":"2024-08-17T02:22:52.941627+0000","last_became_active":"2024-08-17T02:22:52.941330+0000","last_became_peered":"2024-08-17T02:22:52.941330+0000","last_unstale":"2024-08-17T02:22:52.941627+0000","last_undegraded":"2024-08-17T02:22:52.941627+0000","last_fullsized":"2024-08-17T02:22:52.941627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:38.069670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.951576+0000","last_change":"2024-08-17T02:22:51.950178+0000","last_active":"2024-08-17T02:22:51.951576+0000","last_peered":"2024-08-17T02:22:51.951576+0000","last_clean":"2024-08-17T02:22:51.951576+0000","last_became_active":"2024-08-17T02:22:51.949874+0000","last_became_peered":"2024-08-17T02:22:51.949874+0000","last_unstale":"2024-08-17T02:22:51.951576+0000","last_undegraded":"2024-08-17T02:22:51.951576+0000","last_fullsized":"2024-08-17T02:22:51.951576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:11.753629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953541+0000","last_change":"2024-08-17T02:22:52.953541+0000","last_active":"2024-08-17T02:22:52.953541+0000","last_peered":"2024-08-17T02:22:52.953541+0000","last_clean":"2024-08-17T02:22:52.953541+0000","last_became_active":"2024-08-17T02:22:52.953230+0000","last_became_peered":"2024-08-17T02:22:52.953230+0000","last_unstale":"2024-08-17T02:22:52.953541+0000","last_undegraded":"2024-08-17T02:22:52.953541+0000","last_fullsized":"2024-08-17T02:22:52.953541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:38:03.748652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.948769+0000","last_change":"2024-08-17T02:22:51.944266+0000","last_active":"2024-08-17T02:22:51.948769+0000","last_peered":"2024-08-17T02:22:51.948769+0000","last_clean":"2024-08-17T02:22:51.948769+0000","last_became_active":"2024-08-17T02:22:51.943997+0000","last_became_peered":"2024-08-17T02:22:51.943997+0000","last_unstale":"2024-08-17T02:22:51.948769+0000","last_undegraded":"2024-08-17T02:22:51.948769+0000","last_fullsized":"2024-08-17T02:22:51.948769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:30:53.019442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.942986+0000","last_change":"2024-08-17T02:22:52.942986+0000","last_active":"2024-08-17T02:22:52.942986+0000","last_peered":"2024-08-17T02:22:52.942986+0000","last_clean":"2024-08-17T02:22:52.942986+0000","last_became_active":"2024-08-17T02:22:52.942572+0000","last_became_peered":"2024-08-17T02:22:52.942572+0000","last_unstale":"2024-08-17T02:22:52.942986+0000","last_undegraded":"2024-08-17T02:22:52.942986+0000","last_fullsized":"2024-08-17T02:22:52.942986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:07:17.670198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.948993+0000","last_change":"2024-08-17T02:22:51.943853+0000","last_active":"2024-08-17T02:22:51.948993+0000","last_peered":"2024-08-17T02:22:51.948993+0000","last_clean":"2024-08-17T02:22:51.948993+0000","last_became_active":"2024-08-17T02:22:51.943584+0000","last_became_peered":"2024-08-17T02:22:51.943584+0000","last_unstale":"2024-08-17T02:22:51.948993+0000","last_undegraded":"2024-08-17T02:22:51.948993+0000","last_fullsized":"2024-08-17T02:22:51.948993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:04:15.020460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.944169+0000","last_change":"2024-08-17T02:22:52.944169+0000","last_active":"2024-08-17T02:22:52.944169+0000","last_peered":"2024-08-17T02:22:52.944169+0000","last_clean":"2024-08-17T02:22:52.944169+0000","last_became_active":"2024-08-17T02:22:52.943924+0000","last_became_peered":"2024-08-17T02:22:52.943924+0000","last_unstale":"2024-08-17T02:22:52.944169+0000","last_undegraded":"2024-08-17T02:22:52.944169+0000","last_fullsized":"2024-08-17T02:22:52.944169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:32:29.385023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.949707+0000","last_change":"2024-08-17T02:22:51.946852+0000","last_active":"2024-08-17T02:22:51.949707+0000","last_peered":"2024-08-17T02:22:51.949707+0000","last_clean":"2024-08-17T02:22:51.949707+0000","last_became_active":"2024-08-17T02:22:51.946540+0000","last_became_peered":"2024-08-17T02:22:51.946540+0000","last_unstale":"2024-08-17T02:22:51.949707+0000","last_undegraded":"2024-08-17T02:22:51.949707+0000","last_fullsized":"2024-08-17T02:22:51.949707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:43:27.802447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.960778+0000","last_change":"2024-08-17T02:22:52.960778+0000","last_active":"2024-08-17T02:22:52.960778+0000","last_peered":"2024-08-17T02:22:52.960778+0000","last_clean":"2024-08-17T02:22:52.960778+0000","last_became_active":"2024-08-17T02:22:52.960375+0000","last_became_peered":"2024-08-17T02:22:52.960375+0000","last_unstale":"2024-08-17T02:22:52.960778+0000","last_undegraded":"2024-08-17T02:22:52.960778+0000","last_fullsized":"2024-08-17T02:22:52.960778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:57.502728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.941310+0000","last_change":"2024-08-17T02:22:51.936968+0000","last_active":"2024-08-17T02:22:51.941310+0000","last_peered":"2024-08-17T02:22:51.941310+0000","last_clean":"2024-08-17T02:22:51.941310+0000","last_became_active":"2024-08-17T02:22:51.936636+0000","last_became_peered":"2024-08-17T02:22:51.936636+0000","last_unstale":"2024-08-17T02:22:51.941310+0000","last_undegraded":"2024-08-17T02:22:51.941310+0000","last_fullsized":"2024-08-17T02:22:51.941310+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:19:08.814915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.946307+0000","last_change":"2024-08-17T02:22:52.946307+0000","last_active":"2024-08-17T02:22:52.946307+0000","last_peered":"2024-08-17T02:22:52.946307+0000","last_clean":"2024-08-17T02:22:52.946307+0000","last_became_active":"2024-08-17T02:22:52.945308+0000","last_became_peered":"2024-08-17T02:22:52.945308+0000","last_unstale":"2024-08-17T02:22:52.946307+0000","last_undegraded":"2024-08-17T02:22:52.946307+0000","last_fullsized":"2024-08-17T02:22:52.946307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:42:19.890202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.943827+0000","last_change":"2024-08-17T02:22:51.941081+0000","last_active":"2024-08-17T02:22:51.943827+0000","last_peered":"2024-08-17T02:22:51.943827+0000","last_clean":"2024-08-17T02:22:51.943827+0000","last_became_active":"2024-08-17T02:22:51.940738+0000","last_became_peered":"2024-08-17T02:22:51.940738+0000","last_unstale":"2024-08-17T02:22:51.943827+0000","last_undegraded":"2024-08-17T02:22:51.943827+0000","last_fullsized":"2024-08-17T02:22:51.943827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:42:18.886216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.937365+0000","last_change":"2024-08-17T02:22:52.937365+0000","last_active":"2024-08-17T02:22:52.937365+0000","last_peered":"2024-08-17T02:22:52.937365+0000","last_clean":"2024-08-17T02:22:52.937365+0000","last_became_active":"2024-08-17T02:22:52.937085+0000","last_became_peered":"2024-08-17T02:22:52.937085+0000","last_unstale":"2024-08-17T02:22:52.937365+0000","last_undegraded":"2024-08-17T02:22:52.937365+0000","last_fullsized":"2024-08-17T02:22:52.937365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:23:35.890471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.941174+0000","last_change":"2024-08-17T02:22:51.938416+0000","last_active":"2024-08-17T02:22:51.941174+0000","last_peered":"2024-08-17T02:22:51.941174+0000","last_clean":"2024-08-17T02:22:51.941174+0000","last_became_active":"2024-08-17T02:22:51.938065+0000","last_became_peered":"2024-08-17T02:22:51.938065+0000","last_unstale":"2024-08-17T02:22:51.941174+0000","last_undegraded":"2024-08-17T02:22:51.941174+0000","last_fullsized":"2024-08-17T02:22:51.941174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:43:27.802447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.925029+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:51.925029+0000","last_peered":"2024-08-17T02:22:51.925029+0000","last_clean":"2024-08-17T02:22:51.925029+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:51.925029+0000","last_undegraded":"2024-08-17T02:22:51.925029+0000","last_fullsized":"2024-08-17T02:22:51.925029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":937984,"data_stored":718751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074124,"statfs":{"total":96636764160,"available":96331902976,"internally_reserved":0,"allocated":364544,"data_stored":135750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]}]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":364544,"data_stored":135806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]}]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297692,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074148,"statfs":{"total":96636764160,"available":96331927552,"internally_reserved":0,"allocated":344064,"data_stored":128383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":344064,"data_stored":128383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297696,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074144,"statfs":{"total":96636764160,"available":96331923456,"internally_reserved":0,"allocated":344064,"data_stored":128383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]}]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":344064,"data_stored":128383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298276,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073564,"statfs":{"total":96636764160,"available":96331329536,"internally_reserved":0,"allocated":937984,"data_stored":718751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:55.943 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:22:56.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.043+0000 7f364bfff700 1 Processor -- start 2024-08-17T02:22:56.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.043+0000 7f364bfff700 1 -- start start 2024-08-17T02:22:56.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c0ad8a0 con 0x7f364c155d80 2024-08-17T02:22:56.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c0adb80 con 0x7f364c0ad090 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0ade60 con 0x7f364c0af5f0 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364affd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f364c155d80 0x7f364c0aeee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45810/0 (socket says 172.21.15.100:45810) 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364affd700 1 -- 172.21.15.100:0/136091168 learned_addr learned my addr 172.21.15.100:0/136091168 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f36427fc700 1 --2- 172.21.15.100:0/136091168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364a7fc700 1 -- 172.21.15.100:0/136091168 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003279713 0 0) 0x7f364c0ad8a0 con 0x7f364c155d80 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364a7fc700 1 -- 172.21.15.100:0/136091168 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f362c003570 con 0x7f364c155d80 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f36427fc700 1 -- 172.21.15.100:0/136091168 >> v1:172.21.15.123:6789/0 conn(0x7f364c0ad090 legacy=0x7f364c157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f36427fc700 1 -- 172.21.15.100:0/136091168 >> v1:172.21.15.100:6789/0 conn(0x7f364c155d80 legacy=0x7f364c0aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364a7fc700 1 -- 172.21.15.100:0/136091168 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059311716 0 0) 0x7f364c0adb80 con 0x7f364c0ad090 2024-08-17T02:22:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f36427fc700 1 -- 172.21.15.100:0/136091168 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c0ae140 con 0x7f364c0af5f0 2024-08-17T02:22:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f36427fc700 1 --2- 172.21.15.100:0/136091168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0acb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3634009940 tx=0x7f36340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58737172a46e0592 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364a7fc700 1 -- 172.21.15.100:0/136091168 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400f6b0 con 0x7f364c0af5f0 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364a7fc700 1 -- 172.21.15.100:0/136091168 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400f810 con 0x7f364c0af5f0 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364a7fc700 1 -- 172.21.15.100:0/136091168 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400fb90 con 0x7f364c0af5f0 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 -- 172.21.15.100:0/136091168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 msgr2=0x7f364c0acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 --2- 172.21.15.100:0/136091168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0acb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3634009940 tx=0x7f36340091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 -- 172.21.15.100:0/136091168 shutdown_connections 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 --2- 172.21.15.100:0/136091168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0acb50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 -- 172.21.15.100:0/136091168 >> 172.21.15.100:0/136091168 conn(0x7f364c06e070 msgr2=0x7f364c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 -- 172.21.15.100:0/136091168 shutdown_connections 2024-08-17T02:22:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.047+0000 7f364bfff700 1 -- 172.21.15.100:0/136091168 wait complete. 2024-08-17T02:22:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 Processor -- start 2024-08-17T02:22:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 -- start start 2024-08-17T02:22:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c0ad090 0x7f364c144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c155d80 0x7f364c0b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0afe70 con 0x7f364c155d80 2024-08-17T02:22:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0b0110 con 0x7f364c0ad090 2024-08-17T02:22:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0b03f0 con 0x7f364c0af5f0 2024-08-17T02:22:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c155d80 0x7f364c0b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c155d80 0x7f364c0b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58792/0 (socket says 172.21.15.100:58792) 2024-08-17T02:22:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 -- 172.21.15.100:0/3320054672 learned_addr learned my addr 172.21.15.100:0/3320054672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364affd700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c0ad090 0x7f364c144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f36427fc700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 -- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 msgr2=0x7f364c0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 -- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c0ad090 msgr2=0x7f364c144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c0ad090 0x7f364c144590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 -- 172.21.15.100:0/3320054672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3634009420 con 0x7f364c155d80 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364b7fe700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c155d80 0x7f364c0b1e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f364400c8e0 tx=0x7f36440016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f36437fe700 1 -- 172.21.15.100:0/3320054672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644012630 con 0x7f364c155d80 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f36437fe700 1 -- 172.21.15.100:0/3320054672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36440127b0 con 0x7f364c155d80 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c0b06d0 con 0x7f364c155d80 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f36437fe700 1 -- 172.21.15.100:0/3320054672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644012b70 con 0x7f364c155d80 2024-08-17T02:22:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.051+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c0b0c90 con 0x7f364c155d80 2024-08-17T02:22:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.055+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c0b0e80 con 0x7f364c155d80 2024-08-17T02:22:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.055+0000 7f36437fe700 1 -- 172.21.15.100:0/3320054672 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3644003030 con 0x7f364c155d80 2024-08-17T02:22:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.055+0000 7f36437fe700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3630060860 0x7f3630062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.055+0000 7f36437fe700 1 -- 172.21.15.100:0/3320054672 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f364407cc80 con 0x7f364c155d80 2024-08-17T02:22:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.055+0000 7f364affd700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3630060860 0x7f3630062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.059+0000 7f364affd700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3630060860 0x7f3630062d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f363c007fd0 tx=0x7f363c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.067+0000 7f36437fe700 1 -- 172.21.15.100:0/3320054672 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f364404d030 con 0x7f364c155d80 2024-08-17T02:22:56.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.219+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f364c0aeaf0 con 0x7f3630060860 2024-08-17T02:22:56.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.235+0000 7f36437fe700 1 -- 172.21.15.100:0/3320054672 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264564 (secure 0 0 0) 0x7f364c0aeaf0 con 0x7f3630060860 2024-08-17T02:22:56.242 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:56.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3630060860 msgr2=0x7f3630062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3630060860 0x7f3630062d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f363c007fd0 tx=0x7f363c009040 comp rx=0 tx=0).stop 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c155d80 msgr2=0x7f364c0b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c155d80 0x7f364c0b1e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f364400c8e0 tx=0x7f36440016c0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 shutdown_connections 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3630060860 0x7f3630062d10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c155d80 0x7f364c0b1e10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f364c0af5f0 0x7f364c0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 --2- 172.21.15.100:0/3320054672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c0ad090 0x7f364c144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 >> 172.21.15.100:0/3320054672 conn(0x7f364c06e070 msgr2=0x7f364c1570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 shutdown_connections 2024-08-17T02:22:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.239+0000 7f364bfff700 1 -- 172.21.15.100:0/3320054672 wait complete. 2024-08-17T02:22:56.246 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:22:56.257 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-08-17T02:22:55.238320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2391076,"kb_used_data":3888,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752583644,"statfs":{"total":773094113280,"available":770645651456,"internally_reserved":0,"allocated":3981312,"data_stored":2222590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.331356"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.943974+0000","last_change":"2024-08-17T02:22:52.943974+0000","last_active":"2024-08-17T02:22:52.943974+0000","last_peered":"2024-08-17T02:22:52.943974+0000","last_clean":"2024-08-17T02:22:52.943974+0000","last_became_active":"2024-08-17T02:22:52.943395+0000","last_became_peered":"2024-08-17T02:22:52.943395+0000","last_unstale":"2024-08-17T02:22:52.943974+0000","last_undegraded":"2024-08-17T02:22:52.943974+0000","last_fullsized":"2024-08-17T02:22:52.943974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:18:06.671476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.949465+0000","last_change":"2024-08-17T02:22:52.949465+0000","last_active":"2024-08-17T02:22:52.949465+0000","last_peered":"2024-08-17T02:22:52.949465+0000","last_clean":"2024-08-17T02:22:52.949465+0000","last_became_active":"2024-08-17T02:22:52.949084+0000","last_became_peered":"2024-08-17T02:22:52.949084+0000","last_unstale":"2024-08-17T02:22:52.949465+0000","last_undegraded":"2024-08-17T02:22:52.949465+0000","last_fullsized":"2024-08-17T02:22:52.949465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:11:37.208332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.952481+0000","last_change":"2024-08-17T02:22:52.952481+0000","last_active":"2024-08-17T02:22:52.952481+0000","last_peered":"2024-08-17T02:22:52.952481+0000","last_clean":"2024-08-17T02:22:52.952481+0000","last_became_active":"2024-08-17T02:22:52.952195+0000","last_became_peered":"2024-08-17T02:22:52.952195+0000","last_unstale":"2024-08-17T02:22:52.952481+0000","last_undegraded":"2024-08-17T02:22:52.952481+0000","last_fullsized":"2024-08-17T02:22:52.952481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:55:34.680916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.950385+0000","last_change":"2024-08-17T02:22:52.950385+0000","last_active":"2024-08-17T02:22:52.950385+0000","last_peered":"2024-08-17T02:22:52.950385+0000","last_clean":"2024-08-17T02:22:52.950385+0000","last_became_active":"2024-08-17T02:22:52.950103+0000","last_became_peered":"2024-08-17T02:22:52.950103+0000","last_unstale":"2024-08-17T02:22:52.950385+0000","last_undegraded":"2024-08-17T02:22:52.950385+0000","last_fullsized":"2024-08-17T02:22:52.950385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:35:45.700871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.946427+0000","last_change":"2024-08-17T02:22:52.946427+0000","last_active":"2024-08-17T02:22:52.946427+0000","last_peered":"2024-08-17T02:22:52.946427+0000","last_clean":"2024-08-17T02:22:52.946427+0000","last_became_active":"2024-08-17T02:22:52.946175+0000","last_became_peered":"2024-08-17T02:22:52.946175+0000","last_unstale":"2024-08-17T02:22:52.946427+0000","last_undegraded":"2024-08-17T02:22:52.946427+0000","last_fullsized":"2024-08-17T02:22:52.946427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:04:16.024446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.949343+0000","last_change":"2024-08-17T02:22:52.949343+0000","last_active":"2024-08-17T02:22:52.949343+0000","last_peered":"2024-08-17T02:22:52.949343+0000","last_clean":"2024-08-17T02:22:52.949343+0000","last_became_active":"2024-08-17T02:22:52.949063+0000","last_became_peered":"2024-08-17T02:22:52.949063+0000","last_unstale":"2024-08-17T02:22:52.949343+0000","last_undegraded":"2024-08-17T02:22:52.949343+0000","last_fullsized":"2024-08-17T02:22:52.949343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:14:20.665173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953849+0000","last_change":"2024-08-17T02:22:52.953849+0000","last_active":"2024-08-17T02:22:52.953849+0000","last_peered":"2024-08-17T02:22:52.953849+0000","last_clean":"2024-08-17T02:22:52.953849+0000","last_became_active":"2024-08-17T02:22:52.953238+0000","last_became_peered":"2024-08-17T02:22:52.953238+0000","last_unstale":"2024-08-17T02:22:52.953849+0000","last_undegraded":"2024-08-17T02:22:52.953849+0000","last_fullsized":"2024-08-17T02:22:52.953849+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:50:31.084112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.955074+0000","last_change":"2024-08-17T02:22:52.955074+0000","last_active":"2024-08-17T02:22:52.955074+0000","last_peered":"2024-08-17T02:22:52.955074+0000","last_clean":"2024-08-17T02:22:52.955074+0000","last_became_active":"2024-08-17T02:22:52.954606+0000","last_became_peered":"2024-08-17T02:22:52.954606+0000","last_unstale":"2024-08-17T02:22:52.955074+0000","last_undegraded":"2024-08-17T02:22:52.955074+0000","last_fullsized":"2024-08-17T02:22:52.955074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:54:17.718209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.941513+0000","last_change":"2024-08-17T02:22:52.941513+0000","last_active":"2024-08-17T02:22:52.941513+0000","last_peered":"2024-08-17T02:22:52.941513+0000","last_clean":"2024-08-17T02:22:52.941513+0000","last_became_active":"2024-08-17T02:22:52.940959+0000","last_became_peered":"2024-08-17T02:22:52.940959+0000","last_unstale":"2024-08-17T02:22:52.941513+0000","last_undegraded":"2024-08-17T02:22:52.941513+0000","last_fullsized":"2024-08-17T02:22:52.941513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:38:21.669373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.951057+0000","last_change":"2024-08-17T02:22:52.951057+0000","last_active":"2024-08-17T02:22:52.951057+0000","last_peered":"2024-08-17T02:22:52.951057+0000","last_clean":"2024-08-17T02:22:52.951057+0000","last_became_active":"2024-08-17T02:22:52.950755+0000","last_became_peered":"2024-08-17T02:22:52.950755+0000","last_unstale":"2024-08-17T02:22:52.951057+0000","last_undegraded":"2024-08-17T02:22:52.951057+0000","last_fullsized":"2024-08-17T02:22:52.951057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:00:33.074738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953681+0000","last_change":"2024-08-17T02:22:52.953681+0000","last_active":"2024-08-17T02:22:52.953681+0000","last_peered":"2024-08-17T02:22:52.953681+0000","last_clean":"2024-08-17T02:22:52.953681+0000","last_became_active":"2024-08-17T02:22:52.952851+0000","last_became_peered":"2024-08-17T02:22:52.952851+0000","last_unstale":"2024-08-17T02:22:52.953681+0000","last_undegraded":"2024-08-17T02:22:52.953681+0000","last_fullsized":"2024-08-17T02:22:52.953681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:22:58.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.958900+0000","last_change":"2024-08-17T02:22:52.958900+0000","last_active":"2024-08-17T02:22:52.958900+0000","last_peered":"2024-08-17T02:22:52.958900+0000","last_clean":"2024-08-17T02:22:52.958900+0000","last_became_active":"2024-08-17T02:22:52.958634+0000","last_became_peered":"2024-08-17T02:22:52.958634+0000","last_unstale":"2024-08-17T02:22:52.958900+0000","last_undegraded":"2024-08-17T02:22:52.958900+0000","last_fullsized":"2024-08-17T02:22:52.958900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:42:41.132005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.944003+0000","last_change":"2024-08-17T02:22:52.944003+0000","last_active":"2024-08-17T02:22:52.944003+0000","last_peered":"2024-08-17T02:22:52.944003+0000","last_clean":"2024-08-17T02:22:52.944003+0000","last_became_active":"2024-08-17T02:22:52.943740+0000","last_became_peered":"2024-08-17T02:22:52.943740+0000","last_unstale":"2024-08-17T02:22:52.944003+0000","last_undegraded":"2024-08-17T02:22:52.944003+0000","last_fullsized":"2024-08-17T02:22:52.944003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:33.443356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945541+0000","last_change":"2024-08-17T02:22:52.945541+0000","last_active":"2024-08-17T02:22:52.945541+0000","last_peered":"2024-08-17T02:22:52.945541+0000","last_clean":"2024-08-17T02:22:52.945541+0000","last_became_active":"2024-08-17T02:22:52.945245+0000","last_became_peered":"2024-08-17T02:22:52.945245+0000","last_unstale":"2024-08-17T02:22:52.945541+0000","last_undegraded":"2024-08-17T02:22:52.945541+0000","last_fullsized":"2024-08-17T02:22:52.945541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:21:31.564013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.944148+0000","last_change":"2024-08-17T02:22:52.944148+0000","last_active":"2024-08-17T02:22:52.944148+0000","last_peered":"2024-08-17T02:22:52.944148+0000","last_clean":"2024-08-17T02:22:52.944148+0000","last_became_active":"2024-08-17T02:22:52.943844+0000","last_became_peered":"2024-08-17T02:22:52.943844+0000","last_unstale":"2024-08-17T02:22:52.944148+0000","last_undegraded":"2024-08-17T02:22:52.944148+0000","last_fullsized":"2024-08-17T02:22:52.944148+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:40:12.850376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.961941+0000","last_change":"2024-08-17T02:22:52.961941+0000","last_active":"2024-08-17T02:22:52.961941+0000","last_peered":"2024-08-17T02:22:52.961941+0000","last_clean":"2024-08-17T02:22:52.961941+0000","last_became_active":"2024-08-17T02:22:52.961699+0000","last_became_peered":"2024-08-17T02:22:52.961699+0000","last_unstale":"2024-08-17T02:22:52.961941+0000","last_undegraded":"2024-08-17T02:22:52.961941+0000","last_fullsized":"2024-08-17T02:22:52.961941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:12:53.600940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953876+0000","last_change":"2024-08-17T02:22:52.953876+0000","last_active":"2024-08-17T02:22:52.953876+0000","last_peered":"2024-08-17T02:22:52.953876+0000","last_clean":"2024-08-17T02:22:52.953876+0000","last_became_active":"2024-08-17T02:22:52.953286+0000","last_became_peered":"2024-08-17T02:22:52.953286+0000","last_unstale":"2024-08-17T02:22:52.953876+0000","last_undegraded":"2024-08-17T02:22:52.953876+0000","last_fullsized":"2024-08-17T02:22:52.953876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:55:34.680916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.955931+0000","last_change":"2024-08-17T02:22:52.955931+0000","last_active":"2024-08-17T02:22:52.955931+0000","last_peered":"2024-08-17T02:22:52.955931+0000","last_clean":"2024-08-17T02:22:52.955931+0000","last_became_active":"2024-08-17T02:22:52.955267+0000","last_became_peered":"2024-08-17T02:22:52.955267+0000","last_unstale":"2024-08-17T02:22:52.955931+0000","last_undegraded":"2024-08-17T02:22:52.955931+0000","last_fullsized":"2024-08-17T02:22:52.955931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:37:52.854917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954809+0000","last_change":"2024-08-17T02:22:52.954809+0000","last_active":"2024-08-17T02:22:52.954809+0000","last_peered":"2024-08-17T02:22:52.954809+0000","last_clean":"2024-08-17T02:22:52.954809+0000","last_became_active":"2024-08-17T02:22:52.954508+0000","last_became_peered":"2024-08-17T02:22:52.954508+0000","last_unstale":"2024-08-17T02:22:52.954809+0000","last_undegraded":"2024-08-17T02:22:52.954809+0000","last_fullsized":"2024-08-17T02:22:52.954809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:42:59.680721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.937529+0000","last_change":"2024-08-17T02:22:52.937529+0000","last_active":"2024-08-17T02:22:52.937529+0000","last_peered":"2024-08-17T02:22:52.937529+0000","last_clean":"2024-08-17T02:22:52.937529+0000","last_became_active":"2024-08-17T02:22:52.937209+0000","last_became_peered":"2024-08-17T02:22:52.937209+0000","last_unstale":"2024-08-17T02:22:52.937529+0000","last_undegraded":"2024-08-17T02:22:52.937529+0000","last_fullsized":"2024-08-17T02:22:52.937529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:55:56.713536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.940932+0000","last_change":"2024-08-17T02:22:52.940932+0000","last_active":"2024-08-17T02:22:52.940932+0000","last_peered":"2024-08-17T02:22:52.940932+0000","last_clean":"2024-08-17T02:22:52.940932+0000","last_became_active":"2024-08-17T02:22:52.940511+0000","last_became_peered":"2024-08-17T02:22:52.940511+0000","last_unstale":"2024-08-17T02:22:52.940932+0000","last_undegraded":"2024-08-17T02:22:52.940932+0000","last_fullsized":"2024-08-17T02:22:52.940932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:11.073301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.949598+0000","last_change":"2024-08-17T02:22:52.949598+0000","last_active":"2024-08-17T02:22:52.949598+0000","last_peered":"2024-08-17T02:22:52.949598+0000","last_clean":"2024-08-17T02:22:52.949598+0000","last_became_active":"2024-08-17T02:22:52.949310+0000","last_became_peered":"2024-08-17T02:22:52.949310+0000","last_unstale":"2024-08-17T02:22:52.949598+0000","last_undegraded":"2024-08-17T02:22:52.949598+0000","last_fullsized":"2024-08-17T02:22:52.949598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:21:51.352920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.946416+0000","last_change":"2024-08-17T02:22:52.946416+0000","last_active":"2024-08-17T02:22:52.946416+0000","last_peered":"2024-08-17T02:22:52.946416+0000","last_clean":"2024-08-17T02:22:52.946416+0000","last_became_active":"2024-08-17T02:22:52.946096+0000","last_became_peered":"2024-08-17T02:22:52.946096+0000","last_unstale":"2024-08-17T02:22:52.946416+0000","last_undegraded":"2024-08-17T02:22:52.946416+0000","last_fullsized":"2024-08-17T02:22:52.946416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:58:05.009785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.936651+0000","last_change":"2024-08-17T02:22:52.936651+0000","last_active":"2024-08-17T02:22:52.936651+0000","last_peered":"2024-08-17T02:22:52.936651+0000","last_clean":"2024-08-17T02:22:52.936651+0000","last_became_active":"2024-08-17T02:22:52.936235+0000","last_became_peered":"2024-08-17T02:22:52.936235+0000","last_unstale":"2024-08-17T02:22:52.936651+0000","last_undegraded":"2024-08-17T02:22:52.936651+0000","last_fullsized":"2024-08-17T02:22:52.936651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:52:25.176345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.956048+0000","last_change":"2024-08-17T02:22:52.956048+0000","last_active":"2024-08-17T02:22:52.956048+0000","last_peered":"2024-08-17T02:22:52.956048+0000","last_clean":"2024-08-17T02:22:52.956048+0000","last_became_active":"2024-08-17T02:22:52.955575+0000","last_became_peered":"2024-08-17T02:22:52.955575+0000","last_unstale":"2024-08-17T02:22:52.956048+0000","last_undegraded":"2024-08-17T02:22:52.956048+0000","last_fullsized":"2024-08-17T02:22:52.956048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:21:51.352920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.956101+0000","last_change":"2024-08-17T02:22:52.956101+0000","last_active":"2024-08-17T02:22:52.956101+0000","last_peered":"2024-08-17T02:22:52.956101+0000","last_clean":"2024-08-17T02:22:52.956101+0000","last_became_active":"2024-08-17T02:22:52.955609+0000","last_became_peered":"2024-08-17T02:22:52.955609+0000","last_unstale":"2024-08-17T02:22:52.956101+0000","last_undegraded":"2024-08-17T02:22:52.956101+0000","last_fullsized":"2024-08-17T02:22:52.956101+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:41:45.005621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.951193+0000","last_change":"2024-08-17T02:22:52.951193+0000","last_active":"2024-08-17T02:22:52.951193+0000","last_peered":"2024-08-17T02:22:52.951193+0000","last_clean":"2024-08-17T02:22:52.951193+0000","last_became_active":"2024-08-17T02:22:52.950571+0000","last_became_peered":"2024-08-17T02:22:52.950571+0000","last_unstale":"2024-08-17T02:22:52.951193+0000","last_undegraded":"2024-08-17T02:22:52.951193+0000","last_fullsized":"2024-08-17T02:22:52.951193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:26:14.783070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.941997+0000","last_change":"2024-08-17T02:22:52.941997+0000","last_active":"2024-08-17T02:22:52.941997+0000","last_peered":"2024-08-17T02:22:52.941997+0000","last_clean":"2024-08-17T02:22:52.941997+0000","last_became_active":"2024-08-17T02:22:52.941710+0000","last_became_peered":"2024-08-17T02:22:52.941710+0000","last_unstale":"2024-08-17T02:22:52.941997+0000","last_undegraded":"2024-08-17T02:22:52.941997+0000","last_fullsized":"2024-08-17T02:22:52.941997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:50:31.084112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945964+0000","last_change":"2024-08-17T02:22:52.945964+0000","last_active":"2024-08-17T02:22:52.945964+0000","last_peered":"2024-08-17T02:22:52.945964+0000","last_clean":"2024-08-17T02:22:52.945964+0000","last_became_active":"2024-08-17T02:22:52.945625+0000","last_became_peered":"2024-08-17T02:22:52.945625+0000","last_unstale":"2024-08-17T02:22:52.945964+0000","last_undegraded":"2024-08-17T02:22:52.945964+0000","last_fullsized":"2024-08-17T02:22:52.945964+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:35:50.869077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953625+0000","last_change":"2024-08-17T02:22:52.953625+0000","last_active":"2024-08-17T02:22:52.953625+0000","last_peered":"2024-08-17T02:22:52.953625+0000","last_clean":"2024-08-17T02:22:52.953625+0000","last_became_active":"2024-08-17T02:22:52.952707+0000","last_became_peered":"2024-08-17T02:22:52.952707+0000","last_unstale":"2024-08-17T02:22:52.953625+0000","last_undegraded":"2024-08-17T02:22:52.953625+0000","last_fullsized":"2024-08-17T02:22:52.953625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:42:41.132005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.947674+0000","last_change":"2024-08-17T02:22:52.947674+0000","last_active":"2024-08-17T02:22:52.947674+0000","last_peered":"2024-08-17T02:22:52.947674+0000","last_clean":"2024-08-17T02:22:52.947674+0000","last_became_active":"2024-08-17T02:22:52.943447+0000","last_became_peered":"2024-08-17T02:22:52.943447+0000","last_unstale":"2024-08-17T02:22:52.947674+0000","last_undegraded":"2024-08-17T02:22:52.947674+0000","last_fullsized":"2024-08-17T02:22:52.947674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:42:19.890202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953903+0000","last_change":"2024-08-17T02:22:52.953903+0000","last_active":"2024-08-17T02:22:52.953903+0000","last_peered":"2024-08-17T02:22:52.953903+0000","last_clean":"2024-08-17T02:22:52.953903+0000","last_became_active":"2024-08-17T02:22:52.953196+0000","last_became_peered":"2024-08-17T02:22:52.953196+0000","last_unstale":"2024-08-17T02:22:52.953903+0000","last_undegraded":"2024-08-17T02:22:52.953903+0000","last_fullsized":"2024-08-17T02:22:52.953903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:08:05.776194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.947018+0000","last_change":"2024-08-17T02:22:52.947018+0000","last_active":"2024-08-17T02:22:52.947018+0000","last_peered":"2024-08-17T02:22:52.947018+0000","last_clean":"2024-08-17T02:22:52.947018+0000","last_became_active":"2024-08-17T02:22:52.946756+0000","last_became_peered":"2024-08-17T02:22:52.946756+0000","last_unstale":"2024-08-17T02:22:52.947018+0000","last_undegraded":"2024-08-17T02:22:52.947018+0000","last_fullsized":"2024-08-17T02:22:52.947018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:26:19.802145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.948296+0000","last_change":"2024-08-17T02:22:51.939680+0000","last_active":"2024-08-17T02:22:52.948296+0000","last_peered":"2024-08-17T02:22:52.948296+0000","last_clean":"2024-08-17T02:22:52.948296+0000","last_became_active":"2024-08-17T02:22:51.939012+0000","last_became_peered":"2024-08-17T02:22:51.939012+0000","last_unstale":"2024-08-17T02:22:52.948296+0000","last_undegraded":"2024-08-17T02:22:52.948296+0000","last_fullsized":"2024-08-17T02:22:52.948296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:56:48.264910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.948379+0000","last_change":"2024-08-17T02:22:52.948379+0000","last_active":"2024-08-17T02:22:52.948379+0000","last_peered":"2024-08-17T02:22:52.948379+0000","last_clean":"2024-08-17T02:22:52.948379+0000","last_became_active":"2024-08-17T02:22:52.948103+0000","last_became_peered":"2024-08-17T02:22:52.948103+0000","last_unstale":"2024-08-17T02:22:52.948379+0000","last_undegraded":"2024-08-17T02:22:52.948379+0000","last_fullsized":"2024-08-17T02:22:52.948379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:35:50.869077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.926862+0000","last_change":"2024-08-17T02:22:51.949277+0000","last_active":"2024-08-17T02:22:52.926862+0000","last_peered":"2024-08-17T02:22:52.926862+0000","last_clean":"2024-08-17T02:22:52.926862+0000","last_became_active":"2024-08-17T02:22:51.948908+0000","last_became_peered":"2024-08-17T02:22:51.948908+0000","last_unstale":"2024-08-17T02:22:52.926862+0000","last_undegraded":"2024-08-17T02:22:52.926862+0000","last_fullsized":"2024-08-17T02:22:52.926862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:21:48.264201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.959016+0000","last_change":"2024-08-17T02:22:52.959016+0000","last_active":"2024-08-17T02:22:52.959016+0000","last_peered":"2024-08-17T02:22:52.959016+0000","last_clean":"2024-08-17T02:22:52.959016+0000","last_became_active":"2024-08-17T02:22:52.958715+0000","last_became_peered":"2024-08-17T02:22:52.958715+0000","last_unstale":"2024-08-17T02:22:52.959016+0000","last_undegraded":"2024-08-17T02:22:52.959016+0000","last_fullsized":"2024-08-17T02:22:52.959016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:39:13.398435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.958069+0000","last_change":"2024-08-17T02:22:51.945075+0000","last_active":"2024-08-17T02:22:52.958069+0000","last_peered":"2024-08-17T02:22:52.958069+0000","last_clean":"2024-08-17T02:22:52.958069+0000","last_became_active":"2024-08-17T02:22:51.944843+0000","last_became_peered":"2024-08-17T02:22:51.944843+0000","last_unstale":"2024-08-17T02:22:52.958069+0000","last_undegraded":"2024-08-17T02:22:52.958069+0000","last_fullsized":"2024-08-17T02:22:52.958069+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:44:45.251715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.951387+0000","last_change":"2024-08-17T02:22:52.951387+0000","last_active":"2024-08-17T02:22:52.951387+0000","last_peered":"2024-08-17T02:22:52.951387+0000","last_clean":"2024-08-17T02:22:52.951387+0000","last_became_active":"2024-08-17T02:22:52.950936+0000","last_became_peered":"2024-08-17T02:22:52.950936+0000","last_unstale":"2024-08-17T02:22:52.951387+0000","last_undegraded":"2024-08-17T02:22:52.951387+0000","last_fullsized":"2024-08-17T02:22:52.951387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:31.460732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.957448+0000","last_change":"2024-08-17T02:22:51.944451+0000","last_active":"2024-08-17T02:22:52.957448+0000","last_peered":"2024-08-17T02:22:52.957448+0000","last_clean":"2024-08-17T02:22:52.957448+0000","last_became_active":"2024-08-17T02:22:51.944175+0000","last_became_peered":"2024-08-17T02:22:51.944175+0000","last_unstale":"2024-08-17T02:22:52.957448+0000","last_undegraded":"2024-08-17T02:22:52.957448+0000","last_fullsized":"2024-08-17T02:22:52.957448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:51:36.515524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953946+0000","last_change":"2024-08-17T02:22:52.953946+0000","last_active":"2024-08-17T02:22:52.953946+0000","last_peered":"2024-08-17T02:22:52.953946+0000","last_clean":"2024-08-17T02:22:52.953946+0000","last_became_active":"2024-08-17T02:22:52.953011+0000","last_became_peered":"2024-08-17T02:22:52.953011+0000","last_unstale":"2024-08-17T02:22:52.953946+0000","last_undegraded":"2024-08-17T02:22:52.953946+0000","last_fullsized":"2024-08-17T02:22:52.953946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:52:25.176345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.930033+0000","last_change":"2024-08-17T02:22:51.935261+0000","last_active":"2024-08-17T02:22:52.930033+0000","last_peered":"2024-08-17T02:22:52.930033+0000","last_clean":"2024-08-17T02:22:52.930033+0000","last_became_active":"2024-08-17T02:22:51.934981+0000","last_became_peered":"2024-08-17T02:22:51.934981+0000","last_unstale":"2024-08-17T02:22:52.930033+0000","last_undegraded":"2024-08-17T02:22:52.930033+0000","last_fullsized":"2024-08-17T02:22:52.930033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:03:15.711105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.942836+0000","last_change":"2024-08-17T02:22:52.942836+0000","last_active":"2024-08-17T02:22:52.942836+0000","last_peered":"2024-08-17T02:22:52.942836+0000","last_clean":"2024-08-17T02:22:52.942836+0000","last_became_active":"2024-08-17T02:22:52.941938+0000","last_became_peered":"2024-08-17T02:22:52.941938+0000","last_unstale":"2024-08-17T02:22:52.942836+0000","last_undegraded":"2024-08-17T02:22:52.942836+0000","last_fullsized":"2024-08-17T02:22:52.942836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:54:17.718209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.926530+0000","last_change":"2024-08-17T02:22:51.937290+0000","last_active":"2024-08-17T02:22:52.926530+0000","last_peered":"2024-08-17T02:22:52.926530+0000","last_clean":"2024-08-17T02:22:52.926530+0000","last_became_active":"2024-08-17T02:22:51.937023+0000","last_became_peered":"2024-08-17T02:22:51.937023+0000","last_unstale":"2024-08-17T02:22:52.926530+0000","last_undegraded":"2024-08-17T02:22:52.926530+0000","last_fullsized":"2024-08-17T02:22:52.926530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:45:01.390768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945299+0000","last_change":"2024-08-17T02:22:52.945299+0000","last_active":"2024-08-17T02:22:52.945299+0000","last_peered":"2024-08-17T02:22:52.945299+0000","last_clean":"2024-08-17T02:22:52.945299+0000","last_became_active":"2024-08-17T02:22:52.944972+0000","last_became_peered":"2024-08-17T02:22:52.944972+0000","last_unstale":"2024-08-17T02:22:52.945299+0000","last_undegraded":"2024-08-17T02:22:52.945299+0000","last_fullsized":"2024-08-17T02:22:52.945299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:44:46.255701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.944590+0000","last_change":"2024-08-17T02:22:51.941438+0000","last_active":"2024-08-17T02:22:51.944590+0000","last_peered":"2024-08-17T02:22:51.944590+0000","last_clean":"2024-08-17T02:22:51.944590+0000","last_became_active":"2024-08-17T02:22:51.941115+0000","last_became_peered":"2024-08-17T02:22:51.941115+0000","last_unstale":"2024-08-17T02:22:51.944590+0000","last_undegraded":"2024-08-17T02:22:51.944590+0000","last_fullsized":"2024-08-17T02:22:51.944590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:07:14.226381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.947951+0000","last_change":"2024-08-17T02:22:52.947951+0000","last_active":"2024-08-17T02:22:52.947951+0000","last_peered":"2024-08-17T02:22:52.947951+0000","last_clean":"2024-08-17T02:22:52.947951+0000","last_became_active":"2024-08-17T02:22:52.947619+0000","last_became_peered":"2024-08-17T02:22:52.947619+0000","last_unstale":"2024-08-17T02:22:52.947951+0000","last_undegraded":"2024-08-17T02:22:52.947951+0000","last_fullsized":"2024-08-17T02:22:52.947951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:11.073301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.937553+0000","last_change":"2024-08-17T02:22:51.934206+0000","last_active":"2024-08-17T02:22:51.937553+0000","last_peered":"2024-08-17T02:22:51.937553+0000","last_clean":"2024-08-17T02:22:51.937553+0000","last_became_active":"2024-08-17T02:22:51.933897+0000","last_became_peered":"2024-08-17T02:22:51.933897+0000","last_unstale":"2024-08-17T02:22:51.937553+0000","last_undegraded":"2024-08-17T02:22:51.937553+0000","last_fullsized":"2024-08-17T02:22:51.937553+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:30:53.019442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.949404+0000","last_change":"2024-08-17T02:22:52.949404+0000","last_active":"2024-08-17T02:22:52.949404+0000","last_peered":"2024-08-17T02:22:52.949404+0000","last_clean":"2024-08-17T02:22:52.949404+0000","last_became_active":"2024-08-17T02:22:52.949163+0000","last_became_peered":"2024-08-17T02:22:52.949163+0000","last_unstale":"2024-08-17T02:22:52.949404+0000","last_undegraded":"2024-08-17T02:22:52.949404+0000","last_fullsized":"2024-08-17T02:22:52.949404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:23:35.890471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.943556+0000","last_change":"2024-08-17T02:22:51.938623+0000","last_active":"2024-08-17T02:22:51.943556+0000","last_peered":"2024-08-17T02:22:51.943556+0000","last_clean":"2024-08-17T02:22:51.943556+0000","last_became_active":"2024-08-17T02:22:51.938275+0000","last_became_peered":"2024-08-17T02:22:51.938275+0000","last_unstale":"2024-08-17T02:22:51.943556+0000","last_undegraded":"2024-08-17T02:22:51.943556+0000","last_fullsized":"2024-08-17T02:22:51.943556+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:35:49.865091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954698+0000","last_change":"2024-08-17T02:22:52.954698+0000","last_active":"2024-08-17T02:22:52.954698+0000","last_peered":"2024-08-17T02:22:52.954698+0000","last_clean":"2024-08-17T02:22:52.954698+0000","last_became_active":"2024-08-17T02:22:52.954361+0000","last_became_peered":"2024-08-17T02:22:52.954361+0000","last_unstale":"2024-08-17T02:22:52.954698+0000","last_undegraded":"2024-08-17T02:22:52.954698+0000","last_fullsized":"2024-08-17T02:22:52.954698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:18:13.634439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.950267+0000","last_change":"2024-08-17T02:22:51.949130+0000","last_active":"2024-08-17T02:22:51.950267+0000","last_peered":"2024-08-17T02:22:51.950267+0000","last_clean":"2024-08-17T02:22:51.950267+0000","last_became_active":"2024-08-17T02:22:51.948561+0000","last_became_peered":"2024-08-17T02:22:51.948561+0000","last_unstale":"2024-08-17T02:22:51.950267+0000","last_undegraded":"2024-08-17T02:22:51.950267+0000","last_fullsized":"2024-08-17T02:22:51.950267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:32.439370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.959112+0000","last_change":"2024-08-17T02:22:52.959112+0000","last_active":"2024-08-17T02:22:52.959112+0000","last_peered":"2024-08-17T02:22:52.959112+0000","last_clean":"2024-08-17T02:22:52.959112+0000","last_became_active":"2024-08-17T02:22:52.958781+0000","last_became_peered":"2024-08-17T02:22:52.958781+0000","last_unstale":"2024-08-17T02:22:52.959112+0000","last_undegraded":"2024-08-17T02:22:52.959112+0000","last_fullsized":"2024-08-17T02:22:52.959112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:22:22.781465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.955009+0000","last_change":"2024-08-17T02:22:51.936517+0000","last_active":"2024-08-17T02:22:52.955009+0000","last_peered":"2024-08-17T02:22:52.955009+0000","last_clean":"2024-08-17T02:22:52.955009+0000","last_became_active":"2024-08-17T02:22:51.936066+0000","last_became_peered":"2024-08-17T02:22:51.936066+0000","last_unstale":"2024-08-17T02:22:52.955009+0000","last_undegraded":"2024-08-17T02:22:52.955009+0000","last_fullsized":"2024-08-17T02:22:52.955009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:38:02.744666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.952621+0000","last_change":"2024-08-17T02:22:52.952621+0000","last_active":"2024-08-17T02:22:52.952621+0000","last_peered":"2024-08-17T02:22:52.952621+0000","last_clean":"2024-08-17T02:22:52.952621+0000","last_became_active":"2024-08-17T02:22:52.952328+0000","last_became_peered":"2024-08-17T02:22:52.952328+0000","last_unstale":"2024-08-17T02:22:52.952621+0000","last_undegraded":"2024-08-17T02:22:52.952621+0000","last_fullsized":"2024-08-17T02:22:52.952621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:12:53.600940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.962292+0000","last_change":"2024-08-17T02:22:51.950505+0000","last_active":"2024-08-17T02:22:52.962292+0000","last_peered":"2024-08-17T02:22:52.962292+0000","last_clean":"2024-08-17T02:22:52.962292+0000","last_became_active":"2024-08-17T02:22:51.950210+0000","last_became_peered":"2024-08-17T02:22:51.950210+0000","last_unstale":"2024-08-17T02:22:52.962292+0000","last_undegraded":"2024-08-17T02:22:52.962292+0000","last_fullsized":"2024-08-17T02:22:52.962292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:47:16.297181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.950187+0000","last_change":"2024-08-17T02:22:52.950187+0000","last_active":"2024-08-17T02:22:52.950187+0000","last_peered":"2024-08-17T02:22:52.950187+0000","last_clean":"2024-08-17T02:22:52.950187+0000","last_became_active":"2024-08-17T02:22:52.949903+0000","last_became_peered":"2024-08-17T02:22:52.949903+0000","last_unstale":"2024-08-17T02:22:52.950187+0000","last_undegraded":"2024-08-17T02:22:52.950187+0000","last_fullsized":"2024-08-17T02:22:52.950187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:38:21.669373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945963+0000","last_change":"2024-08-17T02:22:51.935474+0000","last_active":"2024-08-17T02:22:52.945963+0000","last_peered":"2024-08-17T02:22:52.945963+0000","last_clean":"2024-08-17T02:22:52.945963+0000","last_became_active":"2024-08-17T02:22:51.934609+0000","last_became_peered":"2024-08-17T02:22:51.934609+0000","last_unstale":"2024-08-17T02:22:52.945963+0000","last_undegraded":"2024-08-17T02:22:52.945963+0000","last_fullsized":"2024-08-17T02:22:52.945963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:38:07.034881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945501+0000","last_change":"2024-08-17T02:22:52.945501+0000","last_active":"2024-08-17T02:22:52.945501+0000","last_peered":"2024-08-17T02:22:52.945501+0000","last_clean":"2024-08-17T02:22:52.945501+0000","last_became_active":"2024-08-17T02:22:52.944947+0000","last_became_peered":"2024-08-17T02:22:52.944947+0000","last_unstale":"2024-08-17T02:22:52.945501+0000","last_undegraded":"2024-08-17T02:22:52.945501+0000","last_fullsized":"2024-08-17T02:22:52.945501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:03:55.493198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.945899+0000","last_change":"2024-08-17T02:22:51.939608+0000","last_active":"2024-08-17T02:22:51.945899+0000","last_peered":"2024-08-17T02:22:51.945899+0000","last_clean":"2024-08-17T02:22:51.945899+0000","last_became_active":"2024-08-17T02:22:51.939266+0000","last_became_peered":"2024-08-17T02:22:51.939266+0000","last_unstale":"2024-08-17T02:22:51.945899+0000","last_undegraded":"2024-08-17T02:22:51.945899+0000","last_fullsized":"2024-08-17T02:22:51.945899+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:07:16.906696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954900+0000","last_change":"2024-08-17T02:22:52.954900+0000","last_active":"2024-08-17T02:22:52.954900+0000","last_peered":"2024-08-17T02:22:52.954900+0000","last_clean":"2024-08-17T02:22:52.954900+0000","last_became_active":"2024-08-17T02:22:52.954225+0000","last_became_peered":"2024-08-17T02:22:52.954225+0000","last_unstale":"2024-08-17T02:22:52.954900+0000","last_undegraded":"2024-08-17T02:22:52.954900+0000","last_fullsized":"2024-08-17T02:22:52.954900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:40:12.850376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.944569+0000","last_change":"2024-08-17T02:22:51.938614+0000","last_active":"2024-08-17T02:22:51.944569+0000","last_peered":"2024-08-17T02:22:51.944569+0000","last_clean":"2024-08-17T02:22:51.944569+0000","last_became_active":"2024-08-17T02:22:51.938329+0000","last_became_peered":"2024-08-17T02:22:51.938329+0000","last_unstale":"2024-08-17T02:22:51.944569+0000","last_undegraded":"2024-08-17T02:22:51.944569+0000","last_fullsized":"2024-08-17T02:22:51.944569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:18:05.667490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.950581+0000","last_change":"2024-08-17T02:22:52.950581+0000","last_active":"2024-08-17T02:22:52.950581+0000","last_peered":"2024-08-17T02:22:52.950581+0000","last_clean":"2024-08-17T02:22:52.950581+0000","last_became_active":"2024-08-17T02:22:52.950290+0000","last_became_peered":"2024-08-17T02:22:52.950290+0000","last_unstale":"2024-08-17T02:22:52.950581+0000","last_undegraded":"2024-08-17T02:22:52.950581+0000","last_fullsized":"2024-08-17T02:22:52.950581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:44:46.255701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.961980+0000","last_change":"2024-08-17T02:22:51.945073+0000","last_active":"2024-08-17T02:22:52.961980+0000","last_peered":"2024-08-17T02:22:52.961980+0000","last_clean":"2024-08-17T02:22:52.961980+0000","last_became_active":"2024-08-17T02:22:51.944804+0000","last_became_peered":"2024-08-17T02:22:51.944804+0000","last_unstale":"2024-08-17T02:22:52.961980+0000","last_undegraded":"2024-08-17T02:22:52.961980+0000","last_fullsized":"2024-08-17T02:22:52.961980+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:04:22.299573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.939950+0000","last_change":"2024-08-17T02:22:52.939950+0000","last_active":"2024-08-17T02:22:52.939950+0000","last_peered":"2024-08-17T02:22:52.939950+0000","last_clean":"2024-08-17T02:22:52.939950+0000","last_became_active":"2024-08-17T02:22:52.939166+0000","last_became_peered":"2024-08-17T02:22:52.939166+0000","last_unstale":"2024-08-17T02:22:52.939950+0000","last_undegraded":"2024-08-17T02:22:52.939950+0000","last_fullsized":"2024-08-17T02:22:52.939950+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:07:17.670198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.938978+0000","last_change":"2024-08-17T02:22:51.945974+0000","last_active":"2024-08-17T02:22:52.938978+0000","last_peered":"2024-08-17T02:22:52.938978+0000","last_clean":"2024-08-17T02:22:52.938978+0000","last_became_active":"2024-08-17T02:22:51.945291+0000","last_became_peered":"2024-08-17T02:22:51.945291+0000","last_unstale":"2024-08-17T02:22:52.938978+0000","last_undegraded":"2024-08-17T02:22:52.938978+0000","last_fullsized":"2024-08-17T02:22:52.938978+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:47:14.245060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.933656+0000","last_change":"2024-08-17T02:22:52.933656+0000","last_active":"2024-08-17T02:22:52.933656+0000","last_peered":"2024-08-17T02:22:52.933656+0000","last_clean":"2024-08-17T02:22:52.933656+0000","last_became_active":"2024-08-17T02:22:52.933236+0000","last_became_peered":"2024-08-17T02:22:52.933236+0000","last_unstale":"2024-08-17T02:22:52.933656+0000","last_undegraded":"2024-08-17T02:22:52.933656+0000","last_fullsized":"2024-08-17T02:22:52.933656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:22:58.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.946432+0000","last_change":"2024-08-17T02:22:51.933910+0000","last_active":"2024-08-17T02:22:52.946432+0000","last_peered":"2024-08-17T02:22:52.946432+0000","last_clean":"2024-08-17T02:22:52.946432+0000","last_became_active":"2024-08-17T02:22:51.933605+0000","last_became_peered":"2024-08-17T02:22:51.933605+0000","last_unstale":"2024-08-17T02:22:52.946432+0000","last_undegraded":"2024-08-17T02:22:52.946432+0000","last_fullsized":"2024-08-17T02:22:52.946432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:50.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.952363+0000","last_change":"2024-08-17T02:22:52.952363+0000","last_active":"2024-08-17T02:22:52.952363+0000","last_peered":"2024-08-17T02:22:52.952363+0000","last_clean":"2024-08-17T02:22:52.952363+0000","last_became_active":"2024-08-17T02:22:52.951977+0000","last_became_peered":"2024-08-17T02:22:52.951977+0000","last_unstale":"2024-08-17T02:22:52.952363+0000","last_undegraded":"2024-08-17T02:22:52.952363+0000","last_fullsized":"2024-08-17T02:22:52.952363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:07:15.230367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.929534+0000","last_change":"2024-08-17T02:22:51.938548+0000","last_active":"2024-08-17T02:22:52.929534+0000","last_peered":"2024-08-17T02:22:52.929534+0000","last_clean":"2024-08-17T02:22:52.929534+0000","last_became_active":"2024-08-17T02:22:51.938244+0000","last_became_peered":"2024-08-17T02:22:51.938244+0000","last_unstale":"2024-08-17T02:22:52.929534+0000","last_undegraded":"2024-08-17T02:22:52.929534+0000","last_fullsized":"2024-08-17T02:22:52.929534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:13:19.985389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954074+0000","last_change":"2024-08-17T02:22:52.954074+0000","last_active":"2024-08-17T02:22:52.954074+0000","last_peered":"2024-08-17T02:22:52.954074+0000","last_clean":"2024-08-17T02:22:52.954074+0000","last_became_active":"2024-08-17T02:22:52.953748+0000","last_became_peered":"2024-08-17T02:22:52.953748+0000","last_unstale":"2024-08-17T02:22:52.954074+0000","last_undegraded":"2024-08-17T02:22:52.954074+0000","last_fullsized":"2024-08-17T02:22:52.954074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:12.757615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.946965+0000","last_change":"2024-08-17T02:22:51.945137+0000","last_active":"2024-08-17T02:22:51.946965+0000","last_peered":"2024-08-17T02:22:51.946965+0000","last_clean":"2024-08-17T02:22:51.946965+0000","last_became_active":"2024-08-17T02:22:51.944710+0000","last_became_peered":"2024-08-17T02:22:51.944710+0000","last_unstale":"2024-08-17T02:22:51.946965+0000","last_undegraded":"2024-08-17T02:22:51.946965+0000","last_fullsized":"2024-08-17T02:22:51.946965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:56:48.264910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.940442+0000","last_change":"2024-08-17T02:22:52.940442+0000","last_active":"2024-08-17T02:22:52.940442+0000","last_peered":"2024-08-17T02:22:52.940442+0000","last_clean":"2024-08-17T02:22:52.940442+0000","last_became_active":"2024-08-17T02:22:52.940137+0000","last_became_peered":"2024-08-17T02:22:52.940137+0000","last_unstale":"2024-08-17T02:22:52.940442+0000","last_undegraded":"2024-08-17T02:22:52.940442+0000","last_fullsized":"2024-08-17T02:22:52.940442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:09:58.358246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.952840+0000","last_change":"2024-08-17T02:22:51.948906+0000","last_active":"2024-08-17T02:22:52.952840+0000","last_peered":"2024-08-17T02:22:52.952840+0000","last_clean":"2024-08-17T02:22:52.952840+0000","last_became_active":"2024-08-17T02:22:51.948578+0000","last_became_peered":"2024-08-17T02:22:51.948578+0000","last_unstale":"2024-08-17T02:22:52.952840+0000","last_undegraded":"2024-08-17T02:22:52.952840+0000","last_fullsized":"2024-08-17T02:22:52.952840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:11.294023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.943423+0000","last_change":"2024-08-17T02:22:52.943423+0000","last_active":"2024-08-17T02:22:52.943423+0000","last_peered":"2024-08-17T02:22:52.943423+0000","last_clean":"2024-08-17T02:22:52.943423+0000","last_became_active":"2024-08-17T02:22:52.941153+0000","last_became_peered":"2024-08-17T02:22:52.941153+0000","last_unstale":"2024-08-17T02:22:52.943423+0000","last_undegraded":"2024-08-17T02:22:52.943423+0000","last_fullsized":"2024-08-17T02:22:52.943423+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:12.298009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.949666+0000","last_change":"2024-08-17T02:22:51.946026+0000","last_active":"2024-08-17T02:22:51.949666+0000","last_peered":"2024-08-17T02:22:51.949666+0000","last_clean":"2024-08-17T02:22:51.949666+0000","last_became_active":"2024-08-17T02:22:51.945731+0000","last_became_peered":"2024-08-17T02:22:51.945731+0000","last_unstale":"2024-08-17T02:22:51.949666+0000","last_undegraded":"2024-08-17T02:22:51.949666+0000","last_fullsized":"2024-08-17T02:22:51.949666+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:16:14.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.945640+0000","last_change":"2024-08-17T02:22:52.945640+0000","last_active":"2024-08-17T02:22:52.945640+0000","last_peered":"2024-08-17T02:22:52.945640+0000","last_clean":"2024-08-17T02:22:52.945640+0000","last_became_active":"2024-08-17T02:22:52.945334+0000","last_became_peered":"2024-08-17T02:22:52.945334+0000","last_unstale":"2024-08-17T02:22:52.945640+0000","last_undegraded":"2024-08-17T02:22:52.945640+0000","last_fullsized":"2024-08-17T02:22:52.945640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:21:49.268187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.944942+0000","last_change":"2024-08-17T02:22:51.943706+0000","last_active":"2024-08-17T02:22:51.944942+0000","last_peered":"2024-08-17T02:22:51.944942+0000","last_clean":"2024-08-17T02:22:51.944942+0000","last_became_active":"2024-08-17T02:22:51.943417+0000","last_became_peered":"2024-08-17T02:22:51.943417+0000","last_unstale":"2024-08-17T02:22:51.944942+0000","last_undegraded":"2024-08-17T02:22:51.944942+0000","last_fullsized":"2024-08-17T02:22:51.944942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:43:04.344779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.954567+0000","last_change":"2024-08-17T02:22:52.954567+0000","last_active":"2024-08-17T02:22:52.954567+0000","last_peered":"2024-08-17T02:22:52.954567+0000","last_clean":"2024-08-17T02:22:52.954567+0000","last_became_active":"2024-08-17T02:22:52.954039+0000","last_became_peered":"2024-08-17T02:22:52.954039+0000","last_unstale":"2024-08-17T02:22:52.954567+0000","last_undegraded":"2024-08-17T02:22:52.954567+0000","last_fullsized":"2024-08-17T02:22:52.954567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:38.069670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.941434+0000","last_change":"2024-08-17T02:22:51.939127+0000","last_active":"2024-08-17T02:22:51.941434+0000","last_peered":"2024-08-17T02:22:51.941434+0000","last_clean":"2024-08-17T02:22:51.941434+0000","last_became_active":"2024-08-17T02:22:51.938849+0000","last_became_peered":"2024-08-17T02:22:51.938849+0000","last_unstale":"2024-08-17T02:22:51.941434+0000","last_undegraded":"2024-08-17T02:22:51.941434+0000","last_fullsized":"2024-08-17T02:22:51.941434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:58:04.005799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.938331+0000","last_change":"2024-08-17T02:22:52.938331+0000","last_active":"2024-08-17T02:22:52.938331+0000","last_peered":"2024-08-17T02:22:52.938331+0000","last_clean":"2024-08-17T02:22:52.938331+0000","last_became_active":"2024-08-17T02:22:52.938010+0000","last_became_peered":"2024-08-17T02:22:52.938010+0000","last_unstale":"2024-08-17T02:22:52.938331+0000","last_undegraded":"2024-08-17T02:22:52.938331+0000","last_fullsized":"2024-08-17T02:22:52.938331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:50:24.419251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.958386+0000","last_change":"2024-08-17T02:22:51.946920+0000","last_active":"2024-08-17T02:22:52.958386+0000","last_peered":"2024-08-17T02:22:52.958386+0000","last_clean":"2024-08-17T02:22:52.958386+0000","last_became_active":"2024-08-17T02:22:51.946636+0000","last_became_peered":"2024-08-17T02:22:51.946636+0000","last_unstale":"2024-08-17T02:22:52.958386+0000","last_undegraded":"2024-08-17T02:22:52.958386+0000","last_fullsized":"2024-08-17T02:22:52.958386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:16:14.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.941627+0000","last_change":"2024-08-17T02:22:52.941627+0000","last_active":"2024-08-17T02:22:52.941627+0000","last_peered":"2024-08-17T02:22:52.941627+0000","last_clean":"2024-08-17T02:22:52.941627+0000","last_became_active":"2024-08-17T02:22:52.941330+0000","last_became_peered":"2024-08-17T02:22:52.941330+0000","last_unstale":"2024-08-17T02:22:52.941627+0000","last_undegraded":"2024-08-17T02:22:52.941627+0000","last_fullsized":"2024-08-17T02:22:52.941627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:34:38.069670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.951576+0000","last_change":"2024-08-17T02:22:51.950178+0000","last_active":"2024-08-17T02:22:51.951576+0000","last_peered":"2024-08-17T02:22:51.951576+0000","last_clean":"2024-08-17T02:22:51.951576+0000","last_became_active":"2024-08-17T02:22:51.949874+0000","last_became_peered":"2024-08-17T02:22:51.949874+0000","last_unstale":"2024-08-17T02:22:51.951576+0000","last_undegraded":"2024-08-17T02:22:51.951576+0000","last_fullsized":"2024-08-17T02:22:51.951576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:11.753629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.953541+0000","last_change":"2024-08-17T02:22:52.953541+0000","last_active":"2024-08-17T02:22:52.953541+0000","last_peered":"2024-08-17T02:22:52.953541+0000","last_clean":"2024-08-17T02:22:52.953541+0000","last_became_active":"2024-08-17T02:22:52.953230+0000","last_became_peered":"2024-08-17T02:22:52.953230+0000","last_unstale":"2024-08-17T02:22:52.953541+0000","last_undegraded":"2024-08-17T02:22:52.953541+0000","last_fullsized":"2024-08-17T02:22:52.953541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:38:03.748652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.948769+0000","last_change":"2024-08-17T02:22:51.944266+0000","last_active":"2024-08-17T02:22:51.948769+0000","last_peered":"2024-08-17T02:22:51.948769+0000","last_clean":"2024-08-17T02:22:51.948769+0000","last_became_active":"2024-08-17T02:22:51.943997+0000","last_became_peered":"2024-08-17T02:22:51.943997+0000","last_unstale":"2024-08-17T02:22:51.948769+0000","last_undegraded":"2024-08-17T02:22:51.948769+0000","last_fullsized":"2024-08-17T02:22:51.948769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:30:53.019442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.942986+0000","last_change":"2024-08-17T02:22:52.942986+0000","last_active":"2024-08-17T02:22:52.942986+0000","last_peered":"2024-08-17T02:22:52.942986+0000","last_clean":"2024-08-17T02:22:52.942986+0000","last_became_active":"2024-08-17T02:22:52.942572+0000","last_became_peered":"2024-08-17T02:22:52.942572+0000","last_unstale":"2024-08-17T02:22:52.942986+0000","last_undegraded":"2024-08-17T02:22:52.942986+0000","last_fullsized":"2024-08-17T02:22:52.942986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:07:17.670198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.948993+0000","last_change":"2024-08-17T02:22:51.943853+0000","last_active":"2024-08-17T02:22:51.948993+0000","last_peered":"2024-08-17T02:22:51.948993+0000","last_clean":"2024-08-17T02:22:51.948993+0000","last_became_active":"2024-08-17T02:22:51.943584+0000","last_became_peered":"2024-08-17T02:22:51.943584+0000","last_unstale":"2024-08-17T02:22:51.948993+0000","last_undegraded":"2024-08-17T02:22:51.948993+0000","last_fullsized":"2024-08-17T02:22:51.948993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:04:15.020460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.944169+0000","last_change":"2024-08-17T02:22:52.944169+0000","last_active":"2024-08-17T02:22:52.944169+0000","last_peered":"2024-08-17T02:22:52.944169+0000","last_clean":"2024-08-17T02:22:52.944169+0000","last_became_active":"2024-08-17T02:22:52.943924+0000","last_became_peered":"2024-08-17T02:22:52.943924+0000","last_unstale":"2024-08-17T02:22:52.944169+0000","last_undegraded":"2024-08-17T02:22:52.944169+0000","last_fullsized":"2024-08-17T02:22:52.944169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:32:29.385023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.949707+0000","last_change":"2024-08-17T02:22:51.946852+0000","last_active":"2024-08-17T02:22:51.949707+0000","last_peered":"2024-08-17T02:22:51.949707+0000","last_clean":"2024-08-17T02:22:51.949707+0000","last_became_active":"2024-08-17T02:22:51.946540+0000","last_became_peered":"2024-08-17T02:22:51.946540+0000","last_unstale":"2024-08-17T02:22:51.949707+0000","last_undegraded":"2024-08-17T02:22:51.949707+0000","last_fullsized":"2024-08-17T02:22:51.949707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:43:27.802447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.960778+0000","last_change":"2024-08-17T02:22:52.960778+0000","last_active":"2024-08-17T02:22:52.960778+0000","last_peered":"2024-08-17T02:22:52.960778+0000","last_clean":"2024-08-17T02:22:52.960778+0000","last_became_active":"2024-08-17T02:22:52.960375+0000","last_became_peered":"2024-08-17T02:22:52.960375+0000","last_unstale":"2024-08-17T02:22:52.960778+0000","last_undegraded":"2024-08-17T02:22:52.960778+0000","last_fullsized":"2024-08-17T02:22:52.960778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:57.502728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.941310+0000","last_change":"2024-08-17T02:22:51.936968+0000","last_active":"2024-08-17T02:22:51.941310+0000","last_peered":"2024-08-17T02:22:51.941310+0000","last_clean":"2024-08-17T02:22:51.941310+0000","last_became_active":"2024-08-17T02:22:51.936636+0000","last_became_peered":"2024-08-17T02:22:51.936636+0000","last_unstale":"2024-08-17T02:22:51.941310+0000","last_undegraded":"2024-08-17T02:22:51.941310+0000","last_fullsized":"2024-08-17T02:22:51.941310+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:19:08.814915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.946307+0000","last_change":"2024-08-17T02:22:52.946307+0000","last_active":"2024-08-17T02:22:52.946307+0000","last_peered":"2024-08-17T02:22:52.946307+0000","last_clean":"2024-08-17T02:22:52.946307+0000","last_became_active":"2024-08-17T02:22:52.945308+0000","last_became_peered":"2024-08-17T02:22:52.945308+0000","last_unstale":"2024-08-17T02:22:52.946307+0000","last_undegraded":"2024-08-17T02:22:52.946307+0000","last_fullsized":"2024-08-17T02:22:52.946307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:42:19.890202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.943827+0000","last_change":"2024-08-17T02:22:51.941081+0000","last_active":"2024-08-17T02:22:51.943827+0000","last_peered":"2024-08-17T02:22:51.943827+0000","last_clean":"2024-08-17T02:22:51.943827+0000","last_became_active":"2024-08-17T02:22:51.940738+0000","last_became_peered":"2024-08-17T02:22:51.940738+0000","last_unstale":"2024-08-17T02:22:51.943827+0000","last_undegraded":"2024-08-17T02:22:51.943827+0000","last_fullsized":"2024-08-17T02:22:51.943827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:42:18.886216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:22:52.937365+0000","last_change":"2024-08-17T02:22:52.937365+0000","last_active":"2024-08-17T02:22:52.937365+0000","last_peered":"2024-08-17T02:22:52.937365+0000","last_clean":"2024-08-17T02:22:52.937365+0000","last_became_active":"2024-08-17T02:22:52.937085+0000","last_became_peered":"2024-08-17T02:22:52.937085+0000","last_unstale":"2024-08-17T02:22:52.937365+0000","last_undegraded":"2024-08-17T02:22:52.937365+0000","last_fullsized":"2024-08-17T02:22:52.937365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:51.907157+0000","last_clean_scrub_stamp":"2024-08-17T02:22:51.907157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:23:35.890471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.941174+0000","last_change":"2024-08-17T02:22:51.938416+0000","last_active":"2024-08-17T02:22:51.941174+0000","last_peered":"2024-08-17T02:22:51.941174+0000","last_clean":"2024-08-17T02:22:51.941174+0000","last_became_active":"2024-08-17T02:22:51.938065+0000","last_became_peered":"2024-08-17T02:22:51.938065+0000","last_unstale":"2024-08-17T02:22:51.941174+0000","last_undegraded":"2024-08-17T02:22:51.941174+0000","last_fullsized":"2024-08-17T02:22:51.941174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:22:50.903171+0000","last_clean_scrub_stamp":"2024-08-17T02:22:50.903171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:43:27.802447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:22:51.925029+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:22:51.925029+0000","last_peered":"2024-08-17T02:22:51.925029+0000","last_clean":"2024-08-17T02:22:51.925029+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:22:51.925029+0000","last_undegraded":"2024-08-17T02:22:51.925029+0000","last_fullsized":"2024-08-17T02:22:51.925029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":937984,"data_stored":718751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074124,"statfs":{"total":96636764160,"available":96331902976,"internally_reserved":0,"allocated":364544,"data_stored":135750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]}]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":364544,"data_stored":135806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]}]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297692,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074148,"statfs":{"total":96636764160,"available":96331927552,"internally_reserved":0,"allocated":344064,"data_stored":128383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":344064,"data_stored":128383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297696,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074144,"statfs":{"total":96636764160,"available":96331923456,"internally_reserved":0,"allocated":344064,"data_stored":128383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]}]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":344064,"data_stored":128383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298276,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073564,"statfs":{"total":96636764160,"available":96331329536,"internally_reserved":0,"allocated":937984,"data_stored":718751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:22:56.261 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-17T02:22:56.261 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-08-17T02:22:56.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b8206a700 1 Processor -- start 2024-08-17T02:22:56.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b8206a700 1 -- start start 2024-08-17T02:22:56.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b8206a700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1b7c0b3c00 0x7f1b7c0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b8206a700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7c0b1ca0 con 0x7f1b7c0b2130 2024-08-17T02:22:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b8206a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7c0b4550 con 0x7f1b7c159110 2024-08-17T02:22:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b8206a700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7c0b1090 con 0x7f1b7c0b3c00 2024-08-17T02:22:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7affd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1b7c0b3c00 0x7f1b7c0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7bfff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1b7c0b2130 0x7f1b7c0b2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45824/0 (socket says 172.21.15.100:45824) 2024-08-17T02:22:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7bfff700 1 -- 172.21.15.100:0/3762720773 learned_addr learned my addr 172.21.15.100:0/3762720773 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7affd700 1 -- 172.21.15.100:0/3762720773 >> v1:172.21.15.123:6789/0 conn(0x7f1b7c159110 legacy=0x7f1b7c0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7affd700 1 -- 172.21.15.100:0/3762720773 >> v1:172.21.15.100:6789/0 conn(0x7f1b7c0b2130 legacy=0x7f1b7c0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7a7fc700 1 -- 172.21.15.100:0/3762720773 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809893478 0 0) 0x7f1b7c0b4550 con 0x7f1b7c159110 2024-08-17T02:22:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7affd700 1 -- 172.21.15.100:0/3762720773 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c0b1370 con 0x7f1b7c0b3c00 2024-08-17T02:22:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7affd700 1 --2- 172.21.15.100:0/3762720773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c0b3c00 0x7f1b7c0b4010 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1b640098e0 tx=0x7f1b640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0ac5a272400b69 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7a7fc700 1 -- 172.21.15.100:0/3762720773 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64009e80 con 0x7f1b7c0b3c00 2024-08-17T02:22:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7a7fc700 1 -- 172.21.15.100:0/3762720773 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6400b760 con 0x7f1b7c0b3c00 2024-08-17T02:22:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.383+0000 7f1b7a7fc700 1 -- 172.21.15.100:0/3762720773 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400bae0 con 0x7f1b7c0b3c00 2024-08-17T02:22:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 -- 172.21.15.100:0/3762720773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c0b3c00 msgr2=0x7f1b7c0b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 --2- 172.21.15.100:0/3762720773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c0b3c00 0x7f1b7c0b4010 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1b640098e0 tx=0x7f1b640091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 -- 172.21.15.100:0/3762720773 shutdown_connections 2024-08-17T02:22:56.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 --2- 172.21.15.100:0/3762720773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c0b3c00 0x7f1b7c0b4010 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 -- 172.21.15.100:0/3762720773 >> 172.21.15.100:0/3762720773 conn(0x7f1b7c0994f0 msgr2=0x7f1b7c067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:56.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 -- 172.21.15.100:0/3762720773 shutdown_connections 2024-08-17T02:22:56.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 -- 172.21.15.100:0/3762720773 wait complete. 2024-08-17T02:22:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 Processor -- start 2024-08-17T02:22:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.387+0000 7f1b8206a700 1 -- start start 2024-08-17T02:22:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b8206a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0b2130 0x7f1b7c09aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b8206a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b7c0b3c00 0x7f1b7c09b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b8206a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c159110 0x7f1b7c169300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b8206a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c15c1e0 con 0x7f1b7c159110 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b8206a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c15bc10 con 0x7f1b7c0b2130 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b8206a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c15ab70 con 0x7f1b7c0b3c00 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c159110 0x7f1b7c169300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c159110 0x7f1b7c169300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58806/0 (socket says 172.21.15.100:58806) 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 -- 172.21.15.100:0/3700913676 learned_addr learned my addr 172.21.15.100:0/3700913676 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7affd700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b7c0b3c00 0x7f1b7c09b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7b7fe700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0b2130 0x7f1b7c09aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 -- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b7c0b3c00 msgr2=0x7f1b7c09b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b7c0b3c00 0x7f1b7c09b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 -- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0b2130 msgr2=0x7f1b7c09aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0b2130 0x7f1b7c09aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 -- 172.21.15.100:0/3700913676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64009420 con 0x7f1b7c159110 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b7bfff700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c159110 0x7f1b7c169300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400cd80 tx=0x7f1b74001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b737fe700 1 -- 172.21.15.100:0/3700913676 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400ce80 con 0x7f1b7c159110 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b737fe700 1 -- 172.21.15.100:0/3700913676 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7400aa80 con 0x7f1b7c159110 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c15ae50 con 0x7f1b7c159110 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b737fe700 1 -- 172.21.15.100:0/3700913676 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400ae20 con 0x7f1b7c159110 2024-08-17T02:22:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.391+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c15b410 con 0x7f1b7c159110 2024-08-17T02:22:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.395+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c0645c0 con 0x7f1b7c159110 2024-08-17T02:22:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.395+0000 7f1b737fe700 1 -- 172.21.15.100:0/3700913676 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1b74007300 con 0x7f1b7c159110 2024-08-17T02:22:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.395+0000 7f1b737fe700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1b5c060860 0x7f1b5c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.395+0000 7f1b737fe700 1 -- 172.21.15.100:0/3700913676 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1b7407cf80 con 0x7f1b7c159110 2024-08-17T02:22:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.395+0000 7f1b7b7fe700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1b5c060860 0x7f1b5c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.395+0000 7f1b7b7fe700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1b5c060860 0x7f1b5c062d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0011e0 tx=0x7f1b6c007ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.403+0000 7f1b737fe700 1 -- 172.21.15.100:0/3700913676 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1b74050e40 con 0x7f1b7c159110 2024-08-17T02:22:56.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.619+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1b7c0af310 con 0x7f1b7c159110 2024-08-17T02:22:56.630 INFO:tasks.ceph.mds.a.smithi100.stderr:2024-08-17T02:22:56.627+0000 7fcf7c5e2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:22:56.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.627+0000 7f1b737fe700 1 -- 172.21.15.100:0/3700913676 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1b740509d0 con 0x7f1b7c159110 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1b5c060860 msgr2=0x7f1b5c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1b5c060860 0x7f1b5c062d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0011e0 tx=0x7f1b6c007ce0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c159110 msgr2=0x7f1b7c169300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c159110 0x7f1b7c169300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400cd80 tx=0x7f1b74001ab0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 shutdown_connections 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1b5c060860 0x7f1b5c062d10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b7c159110 0x7f1b7c169300 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b7c0b3c00 0x7f1b7c09b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 --2- 172.21.15.100:0/3700913676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0b2130 0x7f1b7c09aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 >> 172.21.15.100:0/3700913676 conn(0x7f1b7c0994f0 msgr2=0x7f1b7c156870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:56.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 shutdown_connections 2024-08-17T02:22:56.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.631+0000 7f1b8206a700 1 -- 172.21.15.100:0/3700913676 wait complete. 2024-08-17T02:22:56.635 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-08-17T02:22:56.646 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:22:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd6b6f700 1 Processor -- start 2024-08-17T02:22:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd6b6f700 1 -- start start 2024-08-17T02:22:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd6b6f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd0159040 0x7f6cd00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd6b6f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6cd015d5b0 0x7f6cd00d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd6b6f700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd00cf190 con 0x7f6cd00d0010 2024-08-17T02:22:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd6b6f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd00d09a0 con 0x7f6cd015d5b0 2024-08-17T02:22:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd6b6f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00ce5e0 con 0x7f6cd0159040 2024-08-17T02:22:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd536c700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6cd00d0010 0x7f6cd015cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45830/0 (socket says 172.21.15.100:45830) 2024-08-17T02:22:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd536c700 1 -- 172.21.15.100:0/3746618749 learned_addr learned my addr 172.21.15.100:0/3746618749 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd5b6d700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd0159040 0x7f6cd00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd636e700 1 --2- 172.21.15.100:0/3746618749 >> v2:172.21.15.123:3300/0 conn(0x7f6cd015d5b0 0x7f6cd00d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548645867 0 0) 0x7f6cd00cf190 con 0x7f6cd00d0010 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb8003570 con 0x7f6cd00d0010 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3607064667 0 0) 0x7f6cb8003570 con 0x7f6cd00d0010 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd0159040 msgr2=0x7f6cd00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd0159040 0x7f6cd00cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 >> v2:172.21.15.123:3300/0 conn(0x7f6cd015d5b0 msgr2=0x7f6cd00d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 --2- 172.21.15.100:0/3746618749 >> v2:172.21.15.123:3300/0 conn(0x7f6cd015d5b0 0x7f6cd00d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd00ce860 con 0x7f6cd00d0010 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f6cc80017f0 con 0x7f6cd00d0010 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 >> v1:172.21.15.100:6789/0 conn(0x7f6cd00d0010 legacy=0x7f6cd015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8003a60 0x7f6cb8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd0159040 0x7f6cb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb8004d50 0x7f6cb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00d09a0 con 0x7f6cb8004d50 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00ce5e0 con 0x7f6cb8003a60 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb800ba60 con 0x7f6cd0159040 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd636e700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb8004d50 0x7f6cb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd536c700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd0159040 0x7f6cb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd5b6d700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8003a60 0x7f6cb8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd636e700 1 -- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd0159040 msgr2=0x7f6cb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd636e700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd0159040 0x7f6cb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd636e700 1 -- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8003a60 msgr2=0x7f6cb8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd636e700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8003a60 0x7f6cb8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd636e700 1 -- 172.21.15.100:0/3746618749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb8003570 con 0x7f6cb8004d50 2024-08-17T02:22:56.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd636e700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb8004d50 0x7f6cb80051c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009c00 tx=0x7f6cc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea7392cee98ef1a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000e6b0 con 0x7f6cb8004d50 2024-08-17T02:22:56.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc000e810 con 0x7f6cb8004d50 2024-08-17T02:22:56.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.759+0000 7f6cd4b6b700 1 -- 172.21.15.100:0/3746618749 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000ebd0 con 0x7f6cb8004d50 2024-08-17T02:22:56.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/3746618749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb8004d50 msgr2=0x7f6cb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 --2- 172.21.15.100:0/3746618749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb8004d50 0x7f6cb80051c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009c00 tx=0x7f6cc00091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd636e700 1 -- 172.21.15.100:0/3746618749 reap_dead start 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/3746618749 shutdown_connections 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/3746618749 >> 172.21.15.100:0/3746618749 conn(0x7f6cd0068250 msgr2=0x7f6cd0068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/3746618749 shutdown_connections 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/3746618749 wait complete. 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 Processor -- start 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- start start 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd015c110 0x7f6cd00ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015c4e0 0x7f6cd00b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd0151af0 0x7f6cd0151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd016cbc0 con 0x7f6cd015c4e0 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd01524d0 con 0x7f6cd0151af0 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd6b6f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b2820 con 0x7f6cd015c110 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015c4e0 0x7f6cd00b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015c4e0 0x7f6cd00b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58818/0 (socket says 172.21.15.100:58818) 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 -- 172.21.15.100:0/4155927936 learned_addr learned my addr 172.21.15.100:0/4155927936 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd5b6d700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd015c110 0x7f6cd00ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd636e700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd0151af0 0x7f6cd0151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 -- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd015c110 msgr2=0x7f6cd00ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd015c110 0x7f6cd00ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 -- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd0151af0 msgr2=0x7f6cd0151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd0151af0 0x7f6cd0151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 -- 172.21.15.100:0/4155927936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd00ae660 con 0x7f6cd015c4e0 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd5b6d700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd015c110 0x7f6cd00ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:56.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.763+0000 7f6cd536c700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015c4e0 0x7f6cd00b22e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800a420 tx=0x7f6cc8002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cc5ffb700 1 -- 172.21.15.100:0/4155927936 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800ae50 con 0x7f6cd015c4e0 2024-08-17T02:22:56.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cc5ffb700 1 -- 172.21.15.100:0/4155927936 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc8013600 con 0x7f6cd015c4e0 2024-08-17T02:22:56.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cc5ffb700 1 -- 172.21.15.100:0/4155927936 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8013980 con 0x7f6cd015c4e0 2024-08-17T02:22:56.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd016ce10 con 0x7f6cd015c4e0 2024-08-17T02:22:56.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd016d3d0 con 0x7f6cd015c4e0 2024-08-17T02:22:56.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cc5ffb700 1 -- 172.21.15.100:0/4155927936 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6cc801c400 con 0x7f6cd015c4e0 2024-08-17T02:22:56.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd016d620 con 0x7f6cd015c4e0 2024-08-17T02:22:56.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cc5ffb700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6cbc060890 0x7f6cbc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:56.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cc5ffb700 1 -- 172.21.15.100:0/4155927936 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6cc807dc50 con 0x7f6cd015c4e0 2024-08-17T02:22:56.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cd5b6d700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6cbc060890 0x7f6cbc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:56.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.767+0000 7f6cd5b6d700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6cbc060890 0x7f6cbc062d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc006f00 tx=0x7f6ccc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.775+0000 7f6cc5ffb700 1 -- 172.21.15.100:0/4155927936 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6cc8083020 con 0x7f6cd015c4e0 2024-08-17T02:22:56.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.971+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cd00643a0 con 0x7f6cd015c4e0 2024-08-17T02:22:56.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.971+0000 7f6cc5ffb700 1 -- 172.21.15.100:0/4155927936 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f6cc8017020 con 0x7f6cd015c4e0 2024-08-17T02:22:56.976 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:22:56.977 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:52.296101+0000","modified":"2024-08-17T02:22:54.942750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4980":{"gid":4980,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3037167008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3037167008},{"type":"v1","addr":"172.21.15.100:6835","nonce":3037167008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6cbc060890 msgr2=0x7f6cbc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6cbc060890 0x7f6cbc062d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc006f00 tx=0x7f6ccc008040 comp rx=0 tx=0).stop 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015c4e0 msgr2=0x7f6cd00b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015c4e0 0x7f6cd00b22e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800a420 tx=0x7f6cc8002210 comp rx=0 tx=0).stop 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 shutdown_connections 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6cbc060890 0x7f6cbc062d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd0151af0 0x7f6cd0151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015c4e0 0x7f6cd00b22e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 --2- 172.21.15.100:0/4155927936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cd015c110 0x7f6cd00ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:56.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 >> 172.21.15.100:0/4155927936 conn(0x7f6cd0068250 msgr2=0x7f6cd0153410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:56.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 shutdown_connections 2024-08-17T02:22:56.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:56.975+0000 7f6cd6b6f700 1 -- 172.21.15.100:0/4155927936 wait complete. 2024-08-17T02:22:56.980 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 13 2024-08-17T02:22:57.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-17T02:22:52.296101+0000', 'modified': '2024-08-17T02:22:54.942750+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4980}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4980': {'gid': 4980, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/3037167008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 3037167008}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 3037167008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-17T02:22:57.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-17T02:22:57.394 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-08-17T02:22:57.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.507+0000 7fcef3fff700 1 Processor -- start 2024-08-17T02:22:57.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.507+0000 7fcef3fff700 1 -- start start 2024-08-17T02:22:57.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.507+0000 7fcef3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcef40afac0 0x7fcef40b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.507+0000 7fcef3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.507+0000 7fcef3fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef4069cd0 con 0x7fcef4142a20 2024-08-17T02:22:57.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.507+0000 7fcef3fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40b1700 con 0x7fcef40afac0 2024-08-17T02:22:57.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.507+0000 7fcef3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4068e60 con 0x7fcef40b1280 2024-08-17T02:22:57.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef27fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcef40afac0 0x7fcef40b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4162cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:49720/0 (socket says 172.21.15.100:49720) 2024-08-17T02:22:57.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef2ffd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcef4142a20 0x7fcef4068750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:46734/0 (socket says 172.21.15.100:46734) 2024-08-17T02:22:57.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef37fe700 1 -- 172.21.15.100:0/697694634 learned_addr learned my addr 172.21.15.100:0/697694634 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:57.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef27fc700 1 --2- 172.21.15.100:0/697694634 >> v2:172.21.15.123:3300/0 conn(0x7fcef40afac0 0x7fcef40b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:42230/0 (socket says 172.21.15.100:42230) 2024-08-17T02:22:57.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef37fe700 1 -- 172.21.15.100:0/697694634 >> v2:172.21.15.123:3300/0 conn(0x7fcef40afac0 msgr2=0x7fcef40b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef37fe700 1 --2- 172.21.15.100:0/697694634 >> v2:172.21.15.123:3300/0 conn(0x7fcef40afac0 0x7fcef40b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef37fe700 1 -- 172.21.15.100:0/697694634 >> v1:172.21.15.123:6789/0 conn(0x7fcef4142a20 legacy=0x7fcef4068750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef37fe700 1 -- 172.21.15.100:0/697694634 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40690c0 con 0x7fcef40b1280 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef37fe700 1 --2- 172.21.15.100:0/697694634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4162cd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fceec0099b0 tx=0x7fceec009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f139b41c4493e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef1ffb700 1 -- 172.21.15.100:0/697694634 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec009ea0 con 0x7fcef40b1280 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef1ffb700 1 -- 172.21.15.100:0/697694634 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec00b860 con 0x7fcef40b1280 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef1ffb700 1 -- 172.21.15.100:0/697694634 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec00bc00 con 0x7fcef40b1280 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef3fff700 1 -- 172.21.15.100:0/697694634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 msgr2=0x7fcef4162cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef3fff700 1 --2- 172.21.15.100:0/697694634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4162cd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fceec0099b0 tx=0x7fceec009250 comp rx=0 tx=0).stop 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef3fff700 1 -- 172.21.15.100:0/697694634 shutdown_connections 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef3fff700 1 --2- 172.21.15.100:0/697694634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4162cd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef3fff700 1 --2- 172.21.15.100:0/697694634 >> v2:172.21.15.123:3300/0 conn(0x7fcef40afac0 0x7fcef40b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef3fff700 1 -- 172.21.15.100:0/697694634 >> 172.21.15.100:0/697694634 conn(0x7fcef40cdeb0 msgr2=0x7fcef40ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:57.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef3fff700 1 -- 172.21.15.100:0/697694634 shutdown_connections 2024-08-17T02:22:57.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.511+0000 7fcef3fff700 1 -- 172.21.15.100:0/697694634 wait complete. 2024-08-17T02:22:57.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 Processor -- start 2024-08-17T02:22:57.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 -- start start 2024-08-17T02:22:57.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef40afac0 0x7fcef415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4142a20 0x7fcef413ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4169e80 con 0x7fcef4142a20 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef413f1a0 con 0x7fcef40afac0 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415c200 con 0x7fcef40b1280 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4142a20 0x7fcef413ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4142a20 0x7fcef413ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58828/0 (socket says 172.21.15.100:58828) 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 -- 172.21.15.100:0/1109972214 learned_addr learned my addr 172.21.15.100:0/1109972214 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef27fc700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef40afac0 0x7fcef415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 -- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 msgr2=0x7fcef4169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 -- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef40afac0 msgr2=0x7fcef415bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef40afac0 0x7fcef415bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 -- 172.21.15.100:0/1109972214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0094f0 con 0x7fcef4142a20 2024-08-17T02:22:57.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef27fc700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:22:57.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef37fe700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4142a20 0x7fcef413ec60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcef4068fa0 tx=0x7fceec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fceeaffd700 1 -- 172.21.15.100:0/1109972214 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec011630 con 0x7fcef4142a20 2024-08-17T02:22:57.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 -- 172.21.15.100:0/1109972214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef40c86c0 con 0x7fcef4142a20 2024-08-17T02:22:57.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fceeaffd700 1 -- 172.21.15.100:0/1109972214 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec011790 con 0x7fcef4142a20 2024-08-17T02:22:57.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fcef3fff700 1 -- 172.21.15.100:0/1109972214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef40c8c80 con 0x7fcef4142a20 2024-08-17T02:22:57.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.515+0000 7fceeaffd700 1 -- 172.21.15.100:0/1109972214 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec011b10 con 0x7fcef4142a20 2024-08-17T02:22:57.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.519+0000 7fceeaffd700 1 -- 172.21.15.100:0/1109972214 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fceec011c70 con 0x7fcef4142a20 2024-08-17T02:22:57.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.519+0000 7fceeaffd700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fced4060860 0x7fced4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.519+0000 7fceeaffd700 1 -- 172.21.15.100:0/1109972214 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fceec07e4f0 con 0x7fcef4142a20 2024-08-17T02:22:57.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fced4060860 0x7fced4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fced4060860 0x7fced4062d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcee4007fd0 tx=0x7fcee4007d70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcefb837740 1 Processor -- start 2024-08-17T02:22:57.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcefb837740 1 -- start start 2024-08-17T02:22:57.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcefb837740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x10b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcefb837740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x10b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcefb837740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xecb790 0x10b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcefb837740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x102cba0 con 0xe80710 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcefb837740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1028b40 con 0xe80ae0 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcefb837740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xfee0a0 con 0xecb790 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x10b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xecb790 0x10b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x10b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58834/0 (socket says 172.21.15.100:58834) 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 -- 172.21.15.100:0/97959941 learned_addr learned my addr 172.21.15.100:0/97959941 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:57.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/97959941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x10b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 -- 172.21.15.100:0/97959941 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xecb790 msgr2=0x10b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 --2- 172.21.15.100:0/97959941 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xecb790 0x10b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 -- 172.21.15.100:0/97959941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 msgr2=0x10b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 --2- 172.21.15.100:0/97959941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x10b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 -- 172.21.15.100:0/97959941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xecc9e0 con 0xe80710 2024-08-17T02:22:57.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.523+0000 7fcef27fc700 1 --2- 172.21.15.100:0/97959941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x10b4710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcedc007500 tx=0x7fcedc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3f244365d16fc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fced3fff700 1 -- 172.21.15.100:0/97959941 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00e030 con 0xe80710 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fced3fff700 1 -- 172.21.15.100:0/97959941 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc009c80 con 0xe80710 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fced3fff700 1 -- 172.21.15.100:0/97959941 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc015810 con 0xe80710 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 -- 172.21.15.100:0/97959941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 msgr2=0x10b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 --2- 172.21.15.100:0/97959941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x10b4710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcedc007500 tx=0x7fcedc00b040 comp rx=0 tx=0).stop 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 -- 172.21.15.100:0/97959941 shutdown_connections 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 --2- 172.21.15.100:0/97959941 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xecb790 0x10b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 --2- 172.21.15.100:0/97959941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x10b4710 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 --2- 172.21.15.100:0/97959941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x10b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 -- 172.21.15.100:0/97959941 >> 172.21.15.100:0/97959941 conn(0xe80eb0 msgr2=0x10ba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 -- 172.21.15.100:0/97959941 shutdown_connections 2024-08-17T02:22:57.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 -- 172.21.15.100:0/97959941 wait complete. 2024-08-17T02:22:57.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 -- start start 2024-08-17T02:22:57.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.527+0000 7fcefb837740 1 Processor -- start 2024-08-17T02:22:57.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80710 0x1101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 0x1101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x10bbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0xfedd40 con 0xecb790 2024-08-17T02:22:57.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xfc6970 con 0xe80710 2024-08-17T02:22:57.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xff9690 con 0xe80ae0 2024-08-17T02:22:57.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 10 client.0 _ll_register_callbacks cb 0xfcaf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-17T02:22:57.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x10bbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x10bbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58844/0 (socket says 172.21.15.100:58844) 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 -- 172.21.15.100:0/2631651501 learned_addr learned my addr 172.21.15.100:0/2631651501 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80710 0x1101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef27fc700 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 0x1101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 -- 172.21.15.100:0/2631651501 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 msgr2=0x1101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 0x1101fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 -- 172.21.15.100:0/2631651501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80710 msgr2=0x1101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80710 0x1101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 -- 172.21.15.100:0/2631651501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc009e70 con 0xecb790 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef37fe700 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x10bbfd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fceec01bfd0 tx=0x7fceec07e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14a3439982f37c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fced17fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fced17fa700 1 -- 172.21.15.100:0/2631651501 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec055890 con 0xecb790 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fced17fa700 1 -- 172.21.15.100:0/2631651501 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fceec0595e0 con 0xecb790 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 -- 172.21.15.100:0/2631651501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe8e260 con 0xecb790 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fced17fa700 1 -- 172.21.15.100:0/2631651501 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceec051d00 con 0xecb790 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 10 client.5163 fetch_fsmap learned FSMap version 13 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 -- 172.21.15.100:0/2631651501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0xff6370 con 0xecb790 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fced17fa700 1 -- 172.21.15.100:0/2631651501 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe8e260 con 0xecb790 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fced17fa700 1 -- 172.21.15.100:0/2631651501 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fceec059150 con 0xecb790 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 10 client.5163 fetch_fsmap finished waiting for FSMap version 13 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 10 client.5163 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 20 client.5163 populate_metadata read hostname 'smithi100' 2024-08-17T02:22:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] conn(0x110ac30 0x110d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 4 client.5163 mds_command: new command op to 4980 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-17T02:22:57.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcefb837740 1 -- 172.21.15.100:0/2631651501 --> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xed84e0 con 0x110ac30 2024-08-17T02:22:57.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] conn(0x110ac30 0x110d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] conn(0x110ac30 0x110d010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea5e013ad9d14e98 server_cookie=a0e8e306af80a59f in_seq=0 out_seq=0 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.531+0000 7fced17fa700 10 client.5163 ms_handle_connect on v2:172.21.15.100:6834/3037167008 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fced17fa700 1 -- 172.21.15.100:0/2631651501 <== mds.0 v2:172.21.15.100:6834/3037167008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0xed84e0 con 0x110ac30 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fced17fa700 10 client.5163 handle_command_reply: tid=0 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 client.5163 shutdown 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 -- 172.21.15.100:0/2631651501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 msgr2=0x10bbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x10bbfd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fceec01bfd0 tx=0x7fceec07e040 comp rx=0 tx=0).stop 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 -- 172.21.15.100:0/2631651501 shutdown_connections 2024-08-17T02:22:57.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] conn(0x110ac30 0x110d010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x10bbfd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 0x1101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 --2- 172.21.15.100:0/2631651501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80710 0x1101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fced17fa700 0 client.5163 ms_handle_reset on v2:172.21.15.100:6834/3037167008 2024-08-17T02:22:57.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 -- 172.21.15.100:0/2631651501 >> 172.21.15.100:0/2631651501 conn(0xe80eb0 msgr2=0x10b9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:57.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 -- 172.21.15.100:0/2631651501 shutdown_connections 2024-08-17T02:22:57.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 1 -- 172.21.15.100:0/2631651501 wait complete. 2024-08-17T02:22:57.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.535+0000 7fcefb837740 20 client.5163 trim_cache size 0 max 16384 2024-08-17T02:22:57.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcefb837740 1 Processor -- start 2024-08-17T02:22:57.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcefb837740 1 -- start start 2024-08-17T02:22:57.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcefb837740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x115e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcefb837740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 0x115e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcefb837740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xecb790 0x115edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcefb837740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x1024650 con 0xe80710 2024-08-17T02:22:57.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcefb837740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1000570 con 0xecb790 2024-08-17T02:22:57.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcefb837740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xfd7140 con 0xe80ae0 2024-08-17T02:22:57.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x115e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x115e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58846/0 (socket says 172.21.15.100:58846) 2024-08-17T02:22:57.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xecb790 0x115edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef27fc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 0x115e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 -- 172.21.15.100:0/1684735012 learned_addr learned my addr 172.21.15.100:0/1684735012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 -- 172.21.15.100:0/1684735012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 msgr2=0x115e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/1684735012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 0x115e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 -- 172.21.15.100:0/1684735012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xecb790 msgr2=0x115edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/1684735012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xecb790 0x115edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 -- 172.21.15.100:0/1684735012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xfdcd80 con 0xe80710 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.555+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/1684735012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x115e340 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcee4015ab0 tx=0x7fcee400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5604a5b33fc2c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcee8ff9700 1 -- 172.21.15.100:0/1684735012 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4027220 con 0xe80710 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcee8ff9700 1 -- 172.21.15.100:0/1684735012 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4027860 con 0xe80710 2024-08-17T02:22:57.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcee8ff9700 1 -- 172.21.15.100:0/1684735012 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4027be0 con 0xe80710 2024-08-17T02:22:57.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- 172.21.15.100:0/1684735012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 msgr2=0x115e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 --2- 172.21.15.100:0/1684735012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x115e340 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcee4015ab0 tx=0x7fcee400a000 comp rx=0 tx=0).stop 2024-08-17T02:22:57.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- 172.21.15.100:0/1684735012 shutdown_connections 2024-08-17T02:22:57.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 --2- 172.21.15.100:0/1684735012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xecb790 0x115edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 --2- 172.21.15.100:0/1684735012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80ae0 0x115e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 --2- 172.21.15.100:0/1684735012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xe80710 0x115e340 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- 172.21.15.100:0/1684735012 >> 172.21.15.100:0/1684735012 conn(0xe80eb0 msgr2=0x10ba370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- 172.21.15.100:0/1684735012 shutdown_connections 2024-08-17T02:22:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- 172.21.15.100:0/1684735012 wait complete. 2024-08-17T02:22:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- start start 2024-08-17T02:22:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 Processor -- start 2024-08-17T02:22:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80710 0x11420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x110b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x110b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0xfa1cb0 con 0xecb790 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1028350 con 0xe80ae0 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x10293e0 con 0xe80710 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcefb837740 10 client.0 _ll_register_callbacks cb 0xe4c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x110b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x110b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef2ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80710 0x11420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x110b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:42276/0 (socket says 172.21.15.100:42276) 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef27fc700 1 -- 172.21.15.100:0/2242622608 learned_addr learned my addr 172.21.15.100:0/2242622608 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x110b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58860/0 (socket says 172.21.15.100:58860) 2024-08-17T02:22:57.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef37fe700 1 -- 172.21.15.100:0/2242622608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80710 msgr2=0x11420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef37fe700 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80710 0x11420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef37fe700 1 -- 172.21.15.100:0/2242622608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 msgr2=0x110b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef37fe700 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x110b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef37fe700 1 -- 172.21.15.100:0/2242622608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee400f040 con 0xecb790 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef27fc700 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x110b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fcef37fe700 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x110b540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fceec050950 tx=0x7fceec008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39f1f27b8f3f389 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fced17fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.559+0000 7fced17fa700 1 -- 172.21.15.100:0/2242622608 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0523d0 con 0xecb790 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fced17fa700 1 -- 172.21.15.100:0/2242622608 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fceec0595e0 con 0xecb790 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 -- 172.21.15.100:0/2242622608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcedc009c80 con 0xecb790 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fced17fa700 1 -- 172.21.15.100:0/2242622608 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceec055890 con 0xecb790 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 10 client.5169 fetch_fsmap learned FSMap version 13 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 -- 172.21.15.100:0/2242622608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0xfd1130 con 0xecb790 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fced17fa700 1 -- 172.21.15.100:0/2242622608 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc009c80 con 0xecb790 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fced17fa700 1 -- 172.21.15.100:0/2242622608 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fceec07e040 con 0xecb790 2024-08-17T02:22:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 10 client.5169 fetch_fsmap finished waiting for FSMap version 13 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 10 client.5169 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 20 client.5169 populate_metadata read hostname 'smithi100' 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] conn(0x110ba80 0x1161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 4 client.5169 mds_command: new command op to 4980 tid=0[{"prefix": "status"}] 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 -- 172.21.15.100:0/2242622608 --> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] -- command(tid 0: {"prefix": "status"}) v1 -- 0xefd260 con 0x110ba80 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] conn(0x110ba80 0x1161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fced17fa700 10 client.5169 ms_handle_connect on v2:172.21.15.100:6834/3037167008 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef2ffd700 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] conn(0x110ba80 0x1161d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b7a99296e0cc2 server_cookie=140a926cb3fc70b8 in_seq=0 out_seq=0 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fced17fa700 1 -- 172.21.15.100:0/2242622608 <== mds.0 v2:172.21.15.100:6834/3037167008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0xefd260 con 0x110ba80 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fced17fa700 10 client.5169 handle_command_reply: tid=0 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 client.5169 shutdown 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 -- 172.21.15.100:0/2242622608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 msgr2=0x110b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x110b540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fceec050950 tx=0x7fceec008000 comp rx=0 tx=0).stop 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 -- 172.21.15.100:0/2242622608 shutdown_connections 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.100:6834/3037167008,v1:172.21.15.100:6835/3037167008] conn(0x110ba80 0x1161d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0xecb790 0x110b540 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xe80ae0 0x110b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 --2- 172.21.15.100:0/2242622608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xe80710 0x11420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 -- 172.21.15.100:0/2242622608 >> 172.21.15.100:0/2242622608 conn(0xe80eb0 msgr2=0x1100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fced17fa700 0 client.5169 ms_handle_reset on v2:172.21.15.100:6834/3037167008 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 -- 172.21.15.100:0/2242622608 shutdown_connections 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 1 -- 172.21.15.100:0/2242622608 wait complete. 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcefb837740 20 client.5169 trim_cache size 0 max 16384 2024-08-17T02:22:57.571 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "147b3f96-a106-4648-b442-f3eda3a290c0", 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4980, 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 4.6468201889999996, 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 13, 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 33, 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 33, 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 11.829739274 2024-08-17T02:22:57.572 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:22:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 -- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fced4060860 msgr2=0x7fced4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fced4060860 0x7fced4062d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcee4007fd0 tx=0x7fcee4007d70 comp rx=0 tx=0).stop 2024-08-17T02:22:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 -- 172.21.15.100:0/1109972214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4142a20 msgr2=0x7fcef413ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4142a20 0x7fcef413ec60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcef4068fa0 tx=0x7fceec017000 comp rx=0 tx=0).stop 2024-08-17T02:22:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 -- 172.21.15.100:0/1109972214 shutdown_connections 2024-08-17T02:22:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fced4060860 0x7fced4062d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4142a20 0x7fcef413ec60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcef40b1280 0x7fcef4169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 --2- 172.21.15.100:0/1109972214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef40afac0 0x7fcef415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.563+0000 7fcef3fff700 1 -- 172.21.15.100:0/1109972214 >> 172.21.15.100:0/1109972214 conn(0x7fcef40cdeb0 msgr2=0x7fcef415f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.567+0000 7fcef3fff700 1 -- 172.21.15.100:0/1109972214 shutdown_connections 2024-08-17T02:22:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.567+0000 7fcef3fff700 1 -- 172.21.15.100:0/1109972214 wait complete. 2024-08-17T02:22:57.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:22:57.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.687+0000 7f7e9f59e700 1 Processor -- start 2024-08-17T02:22:57.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.687+0000 7f7e9f59e700 1 -- start start 2024-08-17T02:22:57.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea0068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ea00cc060 0x7f7ea00b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea00cafc0 con 0x7f7ea00caa20 2024-08-17T02:22:57.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea00cb2a0 con 0x7f7ea00cc060 2024-08-17T02:22:57.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00cb580 con 0x7f7ea00688d0 2024-08-17T02:22:57.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e95d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ea00cc060 0x7f7ea00b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9ed9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7ea00caa20 0x7f7ea0156800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:46742/0 (socket says 172.21.15.100:46742) 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9ed9d700 1 -- 172.21.15.100:0/2846551630 learned_addr learned my addr 172.21.15.100:0/2846551630 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9e59c700 1 --2- 172.21.15.100:0/2846551630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea0068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9dd9b700 1 -- 172.21.15.100:0/2846551630 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378631452 0 0) 0x7f7ea00cafc0 con 0x7f7ea00caa20 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9dd9b700 1 -- 172.21.15.100:0/2846551630 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e8c003570 con 0x7f7ea00caa20 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e95d9b700 1 -- 172.21.15.100:0/2846551630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 msgr2=0x7f7ea0068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e95d9b700 1 --2- 172.21.15.100:0/2846551630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea0068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e95d9b700 1 -- 172.21.15.100:0/2846551630 >> v1:172.21.15.123:6789/0 conn(0x7f7ea00caa20 legacy=0x7f7ea0156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e95d9b700 1 -- 172.21.15.100:0/2846551630 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00cb860 con 0x7f7ea00cc060 2024-08-17T02:22:57.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e95d9b700 1 --2- 172.21.15.100:0/2846551630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 0x7f7ea00b1f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e880098e0 tx=0x7f7e880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33e9769f7965b67 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9dd9b700 1 -- 172.21.15.100:0/2846551630 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011630 con 0x7f7ea00cc060 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9dd9b700 1 -- 172.21.15.100:0/2846551630 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e88011790 con 0x7f7ea00cc060 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9dd9b700 1 -- 172.21.15.100:0/2846551630 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011b30 con 0x7f7ea00cc060 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 -- 172.21.15.100:0/2846551630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 msgr2=0x7f7ea00b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/2846551630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 0x7f7ea00b1f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e880098e0 tx=0x7f7e880091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 -- 172.21.15.100:0/2846551630 shutdown_connections 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/2846551630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 0x7f7ea00b1f50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/2846551630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea0068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 -- 172.21.15.100:0/2846551630 >> 172.21.15.100:0/2846551630 conn(0x7f7ea006d790 msgr2=0x7f7ea006db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 -- 172.21.15.100:0/2846551630 shutdown_connections 2024-08-17T02:22:57.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.691+0000 7f7e9f59e700 1 -- 172.21.15.100:0/2846551630 wait complete. 2024-08-17T02:22:57.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9f59e700 1 Processor -- start 2024-08-17T02:22:57.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9f59e700 1 -- start start 2024-08-17T02:22:57.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea00a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9f59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea00caa20 0x7f7ea0155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 0x7f7ea0144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9f59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01427b0 con 0x7f7ea00caa20 2024-08-17T02:22:57.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00a66b0 con 0x7f7ea00cc060 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea0157dd0 con 0x7f7ea00688d0 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e95d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea00caa20 0x7f7ea0155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e95d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea00caa20 0x7f7ea0155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58872/0 (socket says 172.21.15.100:58872) 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e95d9b700 1 -- 172.21.15.100:0/42383525 learned_addr learned my addr 172.21.15.100:0/42383525 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9e59c700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea00a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e9ed9d700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 0x7f7ea0144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e95d9b700 1 -- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 msgr2=0x7f7ea00a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.695+0000 7f7e95d9b700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea00a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e95d9b700 1 -- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 msgr2=0x7f7ea0144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e95d9b700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 0x7f7ea0144d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e95d9b700 1 -- 172.21.15.100:0/42383525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e88009420 con 0x7f7ea00caa20 2024-08-17T02:22:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e9e59c700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea00a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:22:57.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e95d9b700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea00caa20 0x7f7ea0155400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00ac9e0 tx=0x7f7e88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e96ffd700 1 -- 172.21.15.100:0/42383525 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011900 con 0x7f7ea00caa20 2024-08-17T02:22:57.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e96ffd700 1 -- 172.21.15.100:0/42383525 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e88011a60 con 0x7f7ea00caa20 2024-08-17T02:22:57.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea01574a0 con 0x7f7ea00caa20 2024-08-17T02:22:57.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e96ffd700 1 -- 172.21.15.100:0/42383525 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88025620 con 0x7f7ea00caa20 2024-08-17T02:22:57.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea0157a60 con 0x7f7ea00caa20 2024-08-17T02:22:57.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea00643a0 con 0x7f7ea00caa20 2024-08-17T02:22:57.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e96ffd700 1 -- 172.21.15.100:0/42383525 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7e88025a10 con 0x7f7ea00caa20 2024-08-17T02:22:57.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.699+0000 7f7e96ffd700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7e80060860 0x7f7e80062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:22:57.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.703+0000 7f7e96ffd700 1 -- 172.21.15.100:0/42383525 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e8807e1a0 con 0x7f7ea00caa20 2024-08-17T02:22:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.703+0000 7f7e9e59c700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7e80060860 0x7f7e80062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:22:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.703+0000 7f7e9e59c700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7e80060860 0x7f7e80062d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7e900113e0 tx=0x7f7e900091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:22:57.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.711+0000 7f7e96ffd700 1 -- 172.21.15.100:0/42383525 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7e8804e5f0 con 0x7f7ea00caa20 2024-08-17T02:22:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.911+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7ea0157f10 con 0x7f7ea00caa20 2024-08-17T02:22:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.911+0000 7f7e96ffd700 1 -- 172.21.15.100:0/42383525 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7e88051c10 con 0x7f7ea00caa20 2024-08-17T02:22:57.916 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:22:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7e80060860 msgr2=0x7f7e80062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7e80060860 0x7f7e80062d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7e900113e0 tx=0x7f7e900091a0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea00caa20 msgr2=0x7f7ea0155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:22:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea00caa20 0x7f7ea0155400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00ac9e0 tx=0x7f7e88017000 comp rx=0 tx=0).stop 2024-08-17T02:22:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 shutdown_connections 2024-08-17T02:22:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f7e80060860 0x7f7e80062d10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ea00cc060 0x7f7ea0144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea00caa20 0x7f7ea0155400 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 --2- 172.21.15.100:0/42383525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ea00688d0 0x7f7ea00a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:22:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 >> 172.21.15.100:0/42383525 conn(0x7f7ea006d790 msgr2=0x7f7ea006a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:22:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 shutdown_connections 2024-08-17T02:22:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:22:57.915+0000 7f7e9f59e700 1 -- 172.21.15.100:0/42383525 wait complete. 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:22:57.930 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:22:57.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:57.931 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:57.939 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42582sec preferred_lft 42582sec 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:7a:f5:6f:c6 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-17T02:22:57.940 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether be:02:20:d6:7f:e2 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::784b:ceff:fe95:7c38/64 scope link 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether be:02:20:d6:7f:e2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::bc02:20ff:fed6:7fe2/64 scope link 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:22:57.941 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ee:18:7c:7a:d4:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-17T02:22:57.942 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec18:7cff:fe7a:d43f/64 scope link 2024-08-17T02:22:57.942 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:22:57.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:57.942 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:22:57.953 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-08-17T02:22:57.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:57.954 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-08-17T02:22:58.005 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-08-17T02:22:58.005 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:22:58.005 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-17T02:22:58.005 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:22:58.005 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-17T02:22:58.005 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-17T02:22:58.005 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-17T02:22:58.005 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:22:58.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:58.006 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-08-17T02:22:58.006 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-08-17T02:22:58.006 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:22:58.006 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:22:58.006 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:22:58.006 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-08-17T02:22:58.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:58.048 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-08-17T02:22:58.054 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-17T02:22:58.054 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-17T02:22:58.054 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:22:58.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:22:58.054 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-08-17T02:22:58.070 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-17T02:22:58.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:58.071 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:58.119 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:22:58.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:22:58.119 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:58.126 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:58.139 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:58.139 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:58.197 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:58.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:58.197 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:58.202 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-17T02:22:58.203 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-17T02:22:58.205 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:58.219 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:58.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:58.228 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:58.229 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:58.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:58.229 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:58.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-08-17T02:22:58.283+0000 7fe395c9e380 -1 init, newargv = 0x564cbb3299b0 newargc=15 2024-08-17T02:22:58.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[32676]: starting ceph client 2024-08-17T02:22:58.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[32676]: starting fuse 2024-08-17T02:22:59.380 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-08-17T02:22:59.399 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:22:59.399 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:22:59.411 INFO:teuthology.orchestra.run.smithi100.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:22:59.411 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:22:59.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:59.412 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-08-17T02:22:59.464 INFO:teuthology.orchestra.run.smithi100.stdout:52 2024-08-17T02:22:59.465 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-17T02:22:59.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-17T02:22:59.465 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-08-17T02:22:59.466 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-08-17T02:22:59.466 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-08-17T02:22:59.466 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-08-17T02:22:59.466 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:59.467 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-08-17T02:22:59.467 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-17T02:22:59.468 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-08-17T02:22:59.468 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-17T02:22:59.468 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:59.468 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-08-17T02:22:59.468 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-08-17T02:22:59.468 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-08-17T02:22:59.468 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-08-17T02:22:59.468 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-08-17T02:22:59.469 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-08-17T02:22:59.554 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.32676.asok 2024-08-17T02:22:59.741 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.32676.asok 2024-08-17T02:22:59.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:59.742 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.32676.asok status 2024-08-17T02:22:59.838 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:22:59.838 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-08-17T02:22:59.838 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "2577e8139c6b8786f9d6d6ea21b8da9244b9260e", 2024-08-17T02:22:59.838 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 17.2.7-1375-g2577e813 (2577e8139c6b8786f9d6d6ea21b8da9244b9260e) quincy (stable)", 2024-08-17T02:22:59.838 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-08-17T02:22:59.838 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-08-17T02:22:59.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-17T02:22:59.838 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "32676", 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 5178, 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 5178 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-08-17T02:22:59.839 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 848752941 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 848752941 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.5178 192.168.0.1:0/848752941", 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/848752941", 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 13, 2024-08-17T02:22:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 33, 2024-08-17T02:22:59.841 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-08-17T02:22:59.841 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-08-17T02:22:59.841 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-08-17T02:22:59.841 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:22:59.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:59.847 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-08-17T02:22:59.847 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:59.855 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-08-17T02:22:59.855 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:59.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:22:59.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-17T02:22:59.911 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:23:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.023+0000 7fb4d081a700 1 Processor -- start 2024-08-17T02:23:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.023+0000 7fb4d081a700 1 -- start start 2024-08-17T02:23:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.023+0000 7fb4d081a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc1534b0 0x7fb4cc0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.023+0000 7fb4d081a700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0b2990 con 0x7fb4cc159690 2024-08-17T02:23:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.023+0000 7fb4d081a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc159b50 con 0x7fb4cc0b0490 2024-08-17T02:23:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.023+0000 7fb4d081a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc159e30 con 0x7fb4cc1534b0 2024-08-17T02:23:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.023+0000 7fb4caffd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb4cc159690 0x7fb4cc0afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45836/0 (socket says 172.21.15.100:45836) 2024-08-17T02:23:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4caffd700 1 -- 172.21.15.100:0/2599771537 learned_addr learned my addr 172.21.15.100:0/2599771537 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:23:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cb7fe700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc1534b0 0x7fb4cc0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894227184 0 0) 0x7fb4cc0b2990 con 0x7fb4cc159690 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b8003570 con 0x7fb4cc159690 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3694282921 0 0) 0x7fb4b8003570 con 0x7fb4cc159690 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc1534b0 msgr2=0x7fb4cc0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc1534b0 0x7fb4cc0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 >> v1:172.21.15.123:6789/0 conn(0x7fb4cc0b0490 legacy=0x7fb4cc0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc15a110 con 0x7fb4cc159690 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7fb4bc001470 con 0x7fb4cc159690 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 >> v1:172.21.15.100:6789/0 conn(0x7fb4cc159690 legacy=0x7fb4cc0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b8003a90 0x7fb4b8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc1534b0 0x7fb4b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b8004d50 0x7fb4b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc159e30 con 0x7fb4b8004d50 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b800bb10 con 0x7fb4b8003a90 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b800bcd0 con 0x7fb4cc1534b0 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cbfff700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b8004d50 0x7fb4b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4caffd700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc1534b0 0x7fb4b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cb7fe700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b8003a90 0x7fb4b8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cbfff700 1 -- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc1534b0 msgr2=0x7fb4b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cbfff700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc1534b0 0x7fb4b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cbfff700 1 -- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b8003a90 msgr2=0x7fb4b8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cbfff700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b8003a90 0x7fb4b8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cbfff700 1 -- 172.21.15.100:0/2599771537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b8003570 con 0x7fb4b8004d50 2024-08-17T02:23:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cbfff700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b8004d50 0x7fb4b80051c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40020a0 tx=0x7fb4c40016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceae85c48ce99ccb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40019e0 con 0x7fb4b8004d50 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c400c8b0 con 0x7fb4b8004d50 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4ca7fc700 1 -- 172.21.15.100:0/2599771537 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c400cc50 con 0x7fb4b8004d50 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4d081a700 1 -- 172.21.15.100:0/2599771537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b8004d50 msgr2=0x7fb4b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4d081a700 1 --2- 172.21.15.100:0/2599771537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b8004d50 0x7fb4b80051c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40020a0 tx=0x7fb4c40016d0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4cbfff700 1 -- 172.21.15.100:0/2599771537 reap_dead start 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4d081a700 1 -- 172.21.15.100:0/2599771537 shutdown_connections 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4d081a700 1 -- 172.21.15.100:0/2599771537 >> 172.21.15.100:0/2599771537 conn(0x7fb4cc0ca930 msgr2=0x7fb4cc0cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4d081a700 1 -- 172.21.15.100:0/2599771537 shutdown_connections 2024-08-17T02:23:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.027+0000 7fb4d081a700 1 -- 172.21.15.100:0/2599771537 wait complete. 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 Processor -- start 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 -- start start 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4cc06d750 0x7fb4cc153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4cc169b40 0x7fb4cc169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc16b690 0x7fb4cc16bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc16a5f0 con 0x7fb4cc169b40 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc16a8d0 con 0x7fb4cc06d750 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc16abb0 con 0x7fb4cc16b690 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4cc169b40 0x7fb4cc169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4cc169b40 0x7fb4cc169fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58904/0 (socket says 172.21.15.100:58904) 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 -- 172.21.15.100:0/2871510134 learned_addr learned my addr 172.21.15.100:0/2871510134 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4cb7fe700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4cc06d750 0x7fb4cc153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4cbfff700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc16b690 0x7fb4cc16bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 -- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc16b690 msgr2=0x7fb4cc16bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc16b690 0x7fb4cc16bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 -- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4cc06d750 msgr2=0x7fb4cc153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4cc06d750 0x7fb4cc153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 -- 172.21.15.100:0/2871510134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc16ae90 con 0x7fb4cc169b40 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4caffd700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4cc169b40 0x7fb4cc169fb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0073c0 tx=0x7fb4bc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4c37fe700 1 -- 172.21.15.100:0/2871510134 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc018030 con 0x7fb4cc169b40 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4c37fe700 1 -- 172.21.15.100:0/2871510134 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc007850 con 0x7fb4cc169b40 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc16b170 con 0x7fb4cc169b40 2024-08-17T02:23:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4c37fe700 1 -- 172.21.15.100:0/2871510134 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00fac0 con 0x7fb4cc169b40 2024-08-17T02:23:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.031+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc16c040 con 0x7fb4cc169b40 2024-08-17T02:23:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.035+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc16b3b0 con 0x7fb4cc169b40 2024-08-17T02:23:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.035+0000 7fb4c37fe700 1 -- 172.21.15.100:0/2871510134 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb4bc01b070 con 0x7fb4cc169b40 2024-08-17T02:23:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.035+0000 7fb4c37fe700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4ac060890 0x7fb4ac062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.035+0000 7fb4c37fe700 1 -- 172.21.15.100:0/2871510134 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb4bc019070 con 0x7fb4cc169b40 2024-08-17T02:23:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.035+0000 7fb4cb7fe700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4ac060890 0x7fb4ac062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.035+0000 7fb4cb7fe700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4ac060890 0x7fb4ac062d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4006f00 tx=0x7fb4b40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:00.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.043+0000 7fb4c37fe700 1 -- 172.21.15.100:0/2871510134 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb4cc16b3b0 con 0x7fb4cc169b40 2024-08-17T02:23:00.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.243+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4cc0638f0 con 0x7fb4cc169b40 2024-08-17T02:23:00.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.243+0000 7fb4c37fe700 1 -- 172.21.15.100:0/2871510134 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb4bc04cdd0 con 0x7fb4cc169b40 2024-08-17T02:23:00.247 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:23:00.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4ac060890 msgr2=0x7fb4ac062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:00.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4ac060890 0x7fb4ac062d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4006f00 tx=0x7fb4b40091a0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4cc169b40 msgr2=0x7fb4cc169fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4cc169b40 0x7fb4cc169fb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0073c0 tx=0x7fb4bc001e70 comp rx=0 tx=0).stop 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 shutdown_connections 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4ac060890 0x7fb4ac062d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4cc16b690 0x7fb4cc16bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4cc169b40 0x7fb4cc169fb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 --2- 172.21.15.100:0/2871510134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4cc06d750 0x7fb4cc153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 >> 172.21.15.100:0/2871510134 conn(0x7fb4cc0ca930 msgr2=0x7fb4cc152050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 shutdown_connections 2024-08-17T02:23:00.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:00.247+0000 7fb4d081a700 1 -- 172.21.15.100:0/2871510134 wait complete. 2024-08-17T02:23:00.268 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:23:00.268 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:23:00.268 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-08-17T02:23:00.268 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-17T02:23:00.268 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-17T02:23:00.268 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:23:00.268 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:23:00.269 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:23:00.269 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:23:00.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:23:00.269 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:23:00.277 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42577sec preferred_lft 42577sec 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 02:42:7f:c5:f2:c5 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:23:00.278 INFO:teuthology.orchestra.run.smithi123.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 12:60:a8:56:33:42 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::94ae:8dff:fe1e:f582/64 scope link 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 12:60:a8:56:33:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::1060:a8ff:fe56:3342/64 scope link 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether ca:42:c0:4f:64:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::c842:c0ff:fe4f:64a5/64 scope link 2024-08-17T02:23:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:23:00.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:23:00.280 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:23:00.293 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-08-17T02:23:00.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:23:00.294 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-08-17T02:23:00.343 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-08-17T02:23:00.343 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:23:00.343 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-17T02:23:00.343 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:23:00.343 INFO:teuthology.orchestra.run.smithi123.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-17T02:23:00.343 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-17T02:23:00.343 INFO:teuthology.orchestra.run.smithi123.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-17T02:23:00.344 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:23:00.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:23:00.344 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-08-17T02:23:00.344 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-08-17T02:23:00.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:23:00.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:23:00.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:23:00.344 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-08-17T02:23:00.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:23:00.376 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-08-17T02:23:00.383 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-17T02:23:00.383 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-17T02:23:00.383 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:23:00.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:23:00.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-08-17T02:23:00.443 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-08-17T02:23:00.443 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:23:00.444 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-17T02:23:00.449 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:23:00.449 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:23:00.449 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-17T02:23:00.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:23:00.516 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:23:00.516 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:23:00.532 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:23:00.533 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:23:00.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:23:00.533 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:23:00.540 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-17T02:23:00.540 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-08-17T02:23:00.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:23:00.605 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:23:00.605 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:23:00.620 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:23:00.621 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:23:00.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:23:00.622 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:23:00.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:2024-08-17T02:23:00.660+0000 7f9dc801d380 -1 init, newargv = 0x56241e3216b0 newargc=15 2024-08-17T02:23:00.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[29612]: starting ceph client 2024-08-17T02:23:00.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[29612]: starting fuse 2024-08-17T02:23:01.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-08-17T02:23:01.751 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-17T02:23:01.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-17T02:23:01.808 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-17T02:23:01.809 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-17T02:23:01.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:23:01.810 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-08-17T02:23:01.817 INFO:teuthology.orchestra.run.smithi123.stdout:52 2024-08-17T02:23:01.817 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-17T02:23:01.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-17T02:23:01.817 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-08-17T02:23:01.817 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-08-17T02:23:01.818 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-08-17T02:23:01.818 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-08-17T02:23:01.818 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:23:01.818 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-08-17T02:23:01.818 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-17T02:23:01.819 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-17T02:23:01.820 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-17T02:23:01.820 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-08-17T02:23:01.820 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-08-17T02:23:01.820 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-08-17T02:23:01.820 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-17T02:23:01.820 DEBUG:teuthology.orchestra.run.smithi123:> 2024-08-17T02:23:01.820 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.1")) 2024-08-17T02:23:01.820 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-08-17T02:23:01.909 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.1.29612.asok 2024-08-17T02:23:02.092 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29612.asok 2024-08-17T02:23:02.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:23:02.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29612.asok status 2024-08-17T02:23:02.230 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-08-17T02:23:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "2577e8139c6b8786f9d6d6ea21b8da9244b9260e", 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.7-1375-g2577e813 (2577e8139c6b8786f9d6d6ea21b8da9244b9260e) quincy (stable)", 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "1", 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "29612", 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-08-17T02:23:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 5128, 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 5128 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1259253670 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:23:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1259253670 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.5128 192.168.0.1:0/1259253670", 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/1259253670", 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 13, 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 33, 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-08-17T02:23:02.233 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-08-17T02:23:02.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:23:02.240 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-08-17T02:23:02.240 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.1 2024-08-17T02:23:02.246 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-08-17T02:23:02.246 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-08-17T02:23:02.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:23:02.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-17T02:23:02.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-08-17T02:23:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.375+0000 7f832ffff700 1 Processor -- start 2024-08-17T02:23:02.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.375+0000 7f832ffff700 1 -- start start 2024-08-17T02:23:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83300b1480 0x7f8330164170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833006a010 con 0x7f83301427c0 2024-08-17T02:23:02.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8330069490 con 0x7f8330142b90 2024-08-17T02:23:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8330069770 con 0x7f83300b1480 2024-08-17T02:23:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832effd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f83301427c0 0x7f8330068cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45852/0 (socket says 172.21.15.100:45852) 2024-08-17T02:23:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832effd700 1 -- 172.21.15.100:0/2980709264 learned_addr learned my addr 172.21.15.100:0/2980709264 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:23:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832f7fe700 1 --2- 172.21.15.100:0/2980709264 >> v2:172.21.15.123:3300/0 conn(0x7f83300b1480 0x7f8330164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832dffb700 1 -- 172.21.15.100:0/2980709264 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104703081 0 0) 0x7f833006a010 con 0x7f83301427c0 2024-08-17T02:23:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832dffb700 1 -- 172.21.15.100:0/2980709264 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8318003570 con 0x7f83301427c0 2024-08-17T02:23:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832f7fe700 1 -- 172.21.15.100:0/2980709264 >> v1:172.21.15.123:6789/0 conn(0x7f8330142b90 legacy=0x7f83300b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832f7fe700 1 -- 172.21.15.100:0/2980709264 >> v1:172.21.15.100:6789/0 conn(0x7f83301427c0 legacy=0x7f8330068cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832f7fe700 1 -- 172.21.15.100:0/2980709264 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8330069a50 con 0x7f83300b1480 2024-08-17T02:23:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832f7fe700 1 --2- 172.21.15.100:0/2980709264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300b1480 0x7f8330164170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8328009e60 tx=0x7f8328009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435d79644e25954 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832dffb700 1 -- 172.21.15.100:0/2980709264 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800e070 con 0x7f83300b1480 2024-08-17T02:23:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832dffb700 1 -- 172.21.15.100:0/2980709264 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832800bb60 con 0x7f83300b1480 2024-08-17T02:23:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832dffb700 1 -- 172.21.15.100:0/2980709264 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328012430 con 0x7f83300b1480 2024-08-17T02:23:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 -- 172.21.15.100:0/2980709264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300b1480 msgr2=0x7f8330164170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 --2- 172.21.15.100:0/2980709264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300b1480 0x7f8330164170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8328009e60 tx=0x7f8328009780 comp rx=0 tx=0).stop 2024-08-17T02:23:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 -- 172.21.15.100:0/2980709264 shutdown_connections 2024-08-17T02:23:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 --2- 172.21.15.100:0/2980709264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300b1480 0x7f8330164170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 -- 172.21.15.100:0/2980709264 >> 172.21.15.100:0/2980709264 conn(0x7f83300cdea0 msgr2=0x7f83300ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 -- 172.21.15.100:0/2980709264 shutdown_connections 2024-08-17T02:23:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.379+0000 7f832ffff700 1 -- 172.21.15.100:0/2980709264 wait complete. 2024-08-17T02:23:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832ffff700 1 Processor -- start 2024-08-17T02:23:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832ffff700 1 -- start start 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300b1480 0x7f833016d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83301427c0 0x7f83300a7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330142b90 0x7f833013e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832ffff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833016db50 con 0x7f8330142b90 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83300cdc80 con 0x7f83301427c0 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83300cd9a0 con 0x7f83300b1480 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330142b90 0x7f833013e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330142b90 0x7f833013e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58920/0 (socket says 172.21.15.100:58920) 2024-08-17T02:23:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832f7fe700 1 -- 172.21.15.100:0/463783128 learned_addr learned my addr 172.21.15.100:0/463783128 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832effd700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300b1480 0x7f833016d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832e7fc700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83301427c0 0x7f83300a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832f7fe700 1 -- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300b1480 msgr2=0x7f833016d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832f7fe700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300b1480 0x7f833016d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832f7fe700 1 -- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83301427c0 msgr2=0x7f83300a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832f7fe700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83301427c0 0x7f83300a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.383+0000 7f832f7fe700 1 -- 172.21.15.100:0/463783128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328009a20 con 0x7f8330142b90 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f832f7fe700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330142b90 0x7f833013e220 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8328013000 tx=0x7f8328019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f8316ffd700 1 -- 172.21.15.100:0/463783128 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328012d50 con 0x7f8330142b90 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f8316ffd700 1 -- 172.21.15.100:0/463783128 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328021070 con 0x7f8330142b90 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f8316ffd700 1 -- 172.21.15.100:0/463783128 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328026680 con 0x7f8330142b90 2024-08-17T02:23:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83301697f0 con 0x7f8330142b90 2024-08-17T02:23:02.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8330169db0 con 0x7f8330142b90 2024-08-17T02:23:02.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8330169650 con 0x7f8330142b90 2024-08-17T02:23:02.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f8316ffd700 1 -- 172.21.15.100:0/463783128 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8328023030 con 0x7f8330142b90 2024-08-17T02:23:02.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f8316ffd700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f831c060860 0x7f831c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:02.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f8316ffd700 1 -- 172.21.15.100:0/463783128 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f832807ee50 con 0x7f8330142b90 2024-08-17T02:23:02.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.387+0000 7f832effd700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f831c060860 0x7f831c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:02.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.391+0000 7f832effd700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f831c060860 0x7f831c062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83300ced70 tx=0x7f8324009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:02.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.395+0000 7f8316ffd700 1 -- 172.21.15.100:0/463783128 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8330169650 con 0x7f8330142b90 2024-08-17T02:23:02.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.591+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8330169650 con 0x7f8330142b90 2024-08-17T02:23:02.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.979+0000 7f8316ffd700 1 -- 172.21.15.100:0/463783128 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8330169650 con 0x7f8330142b90 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f831c060860 msgr2=0x7f831c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f831c060860 0x7f831c062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83300ced70 tx=0x7f8324009040 comp rx=0 tx=0).stop 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330142b90 msgr2=0x7f833013e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330142b90 0x7f833013e220 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8328013000 tx=0x7f8328019000 comp rx=0 tx=0).stop 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 shutdown_connections 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f831c060860 0x7f831c062d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330142b90 0x7f833013e220 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83301427c0 0x7f83300a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 --2- 172.21.15.100:0/463783128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300b1480 0x7f833016d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 >> 172.21.15.100:0/463783128 conn(0x7f83300cdea0 msgr2=0x7f8330140b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 shutdown_connections 2024-08-17T02:23:02.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:02.983+0000 7f832ffff700 1 -- 172.21.15.100:0/463783128 wait complete. 2024-08-17T02:23:03.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-17T02:23:03.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00dc4d1700 1 Processor -- start 2024-08-17T02:23:03.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00dc4d1700 1 -- start start 2024-08-17T02:23:03.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00dc4d1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00dc4d1700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d40cb890 con 0x7f00d40adf10 2024-08-17T02:23:03.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00dc4d1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d41610c0 con 0x7f00d40cad40 2024-08-17T02:23:03.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00dc4d1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4158290 con 0x7f00d406e470 2024-08-17T02:23:03.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00da26d700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f00d40adf10 0x7f00d40ae2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45866/0 (socket says 172.21.15.100:45866) 2024-08-17T02:23:03.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00da26d700 1 -- 172.21.15.100:0/2404334579 learned_addr learned my addr 172.21.15.100:0/2404334579 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:23:03.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d9a6c700 1 --2- 172.21.15.100:0/2404334579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d926b700 1 -- 172.21.15.100:0/2404334579 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356957297 0 0) 0x7f00d40cb890 con 0x7f00d40adf10 2024-08-17T02:23:03.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d926b700 1 -- 172.21.15.100:0/2404334579 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00c0003570 con 0x7f00d40adf10 2024-08-17T02:23:03.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d926b700 1 -- 172.21.15.100:0/2404334579 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292596845 0 0) 0x7f00d41610c0 con 0x7f00d40cad40 2024-08-17T02:23:03.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d9a6c700 1 -- 172.21.15.100:0/2404334579 >> v1:172.21.15.123:6789/0 conn(0x7f00d40cad40 legacy=0x7f00d40cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d9a6c700 1 -- 172.21.15.100:0/2404334579 >> v1:172.21.15.100:6789/0 conn(0x7f00d40adf10 legacy=0x7f00d40ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d9a6c700 1 -- 172.21.15.100:0/2404334579 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d415dff0 con 0x7f00d406e470 2024-08-17T02:23:03.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d9a6c700 1 --2- 172.21.15.100:0/2404334579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d406e8e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009910 tx=0x7f00c40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc17f65761ff6e0b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d926b700 1 -- 172.21.15.100:0/2404334579 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c400b680 con 0x7f00d406e470 2024-08-17T02:23:03.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d926b700 1 -- 172.21.15.100:0/2404334579 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c400b7e0 con 0x7f00d406e470 2024-08-17T02:23:03.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.107+0000 7f00d926b700 1 -- 172.21.15.100:0/2404334579 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c400bba0 con 0x7f00d406e470 2024-08-17T02:23:03.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 -- 172.21.15.100:0/2404334579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 msgr2=0x7f00d406e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 --2- 172.21.15.100:0/2404334579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d406e8e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009910 tx=0x7f00c40091a0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 -- 172.21.15.100:0/2404334579 shutdown_connections 2024-08-17T02:23:03.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 --2- 172.21.15.100:0/2404334579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d406e8e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 -- 172.21.15.100:0/2404334579 >> 172.21.15.100:0/2404334579 conn(0x7f00d4067810 msgr2=0x7f00d4067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:03.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 -- 172.21.15.100:0/2404334579 shutdown_connections 2024-08-17T02:23:03.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 -- 172.21.15.100:0/2404334579 wait complete. 2024-08-17T02:23:03.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 Processor -- start 2024-08-17T02:23:03.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.111+0000 7f00dc4d1700 1 -- start start 2024-08-17T02:23:03.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00d40adf10 0x7f00d416ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00d40cad40 0x7f00d40aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40b1be0 con 0x7f00d40cad40 2024-08-17T02:23:03.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40a9510 con 0x7f00d40adf10 2024-08-17T02:23:03.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40b1900 con 0x7f00d406e470 2024-08-17T02:23:03.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00d40cad40 0x7f00d40aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00d40cad40 0x7f00d40aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58932/0 (socket says 172.21.15.100:58932) 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 -- 172.21.15.100:0/1399042787 learned_addr learned my addr 172.21.15.100:0/1399042787 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00da26d700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00d9a6c700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00d40adf10 0x7f00d416ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 -- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 msgr2=0x7f00d40b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d40b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 -- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00d40adf10 msgr2=0x7f00d416ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00d40adf10 0x7f00d416ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 -- 172.21.15.100:0/1399042787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c4009420 con 0x7f00d40cad40 2024-08-17T02:23:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00daa6e700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00d40cad40 0x7f00d40aed90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00c8c0 tx=0x7f00cc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00ca7fc700 1 -- 172.21.15.100:0/1399042787 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc012630 con 0x7f00d40cad40 2024-08-17T02:23:03.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00ca7fc700 1 -- 172.21.15.100:0/1399042787 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc012790 con 0x7f00d40cad40 2024-08-17T02:23:03.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d40a8ec0 con 0x7f00d40cad40 2024-08-17T02:23:03.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00ca7fc700 1 -- 172.21.15.100:0/1399042787 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc012b50 con 0x7f00d40cad40 2024-08-17T02:23:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d40af2d0 con 0x7f00d40cad40 2024-08-17T02:23:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.115+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d40a8d20 con 0x7f00d40cad40 2024-08-17T02:23:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.119+0000 7f00ca7fc700 1 -- 172.21.15.100:0/1399042787 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f00cc01a3a0 con 0x7f00d40cad40 2024-08-17T02:23:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.119+0000 7f00ca7fc700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f00b8060860 0x7f00b8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.119+0000 7f00ca7fc700 1 -- 172.21.15.100:0/1399042787 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f00cc07cd30 con 0x7f00d40cad40 2024-08-17T02:23:03.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.119+0000 7f00da26d700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f00b8060860 0x7f00b8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.119+0000 7f00da26d700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f00b8060860 0x7f00b8062d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f00d415b8d0 tx=0x7f00d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.123+0000 7f00ca7fc700 1 -- 172.21.15.100:0/1399042787 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f00d40a8d20 con 0x7f00d40cad40 2024-08-17T02:23:03.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.307+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f00d40a8d20 con 0x7f00b8060860 2024-08-17T02:23:03.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.311+0000 7f00ca7fc700 1 -- 172.21.15.100:0/1399042787 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f00d40a8d20 con 0x7f00b8060860 2024-08-17T02:23:03.315 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-08-17T02:23:03.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:23:03.315 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-08-17T02:23:03.315 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:23:03.315 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-08-17T02:23:03.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f00b8060860 msgr2=0x7f00b8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f00b8060860 0x7f00b8062d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f00d415b8d0 tx=0x7f00d0009040 comp rx=0 tx=0).stop 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00d40cad40 msgr2=0x7f00d40aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00d40cad40 0x7f00d40aed90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00c8c0 tx=0x7f00cc001670 comp rx=0 tx=0).stop 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 shutdown_connections 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f00b8060860 0x7f00b8062d10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00d40cad40 0x7f00d40aed90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00d40adf10 0x7f00d416ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 --2- 172.21.15.100:0/1399042787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00d406e470 0x7f00d40b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 >> 172.21.15.100:0/1399042787 conn(0x7f00d4067810 msgr2=0x7f00d4155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 shutdown_connections 2024-08-17T02:23:03.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.315+0000 7f00dc4d1700 1 -- 172.21.15.100:0/1399042787 wait complete. 2024-08-17T02:23:03.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-17T02:23:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b18887700 1 Processor -- start 2024-08-17T02:23:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b18887700 1 -- start start 2024-08-17T02:23:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b18887700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b140cbab0 0x7f0b140cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b18887700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140cc450 con 0x7f0b140ad6e0 2024-08-17T02:23:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b18887700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140c8c30 con 0x7f0b14140910 2024-08-17T02:23:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b18887700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b140c3e30 con 0x7f0b140cbab0 2024-08-17T02:23:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b1359e700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0b140ad6e0 0x7f0b140adb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45868/0 (socket says 172.21.15.100:45868) 2024-08-17T02:23:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b1359e700 1 -- 172.21.15.100:0/3135030119 learned_addr learned my addr 172.21.15.100:0/3135030119 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:23:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b1259c700 1 --2- 172.21.15.100:0/3135030119 >> v2:172.21.15.123:3300/0 conn(0x7f0b140cbab0 0x7f0b140cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b11d9b700 1 -- 172.21.15.100:0/3135030119 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665569426 0 0) 0x7f0b0c001110 con 0x7f0b140ad6e0 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b11d9b700 1 -- 172.21.15.100:0/3135030119 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b00003570 con 0x7f0b140ad6e0 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b1259c700 1 -- 172.21.15.100:0/3135030119 >> v1:172.21.15.123:6789/0 conn(0x7f0b14140910 legacy=0x7f0b140c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b1259c700 1 -- 172.21.15.100:0/3135030119 >> v1:172.21.15.100:6789/0 conn(0x7f0b140ad6e0 legacy=0x7f0b140adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b1259c700 1 -- 172.21.15.100:0/3135030119 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140c4110 con 0x7f0b140cbab0 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.443+0000 7f0b1259c700 1 --2- 172.21.15.100:0/3135030119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b140cbab0 0x7f0b140cbf10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009e60 tx=0x7f0b04009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7937dd152049ad4d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b11d9b700 1 -- 172.21.15.100:0/3135030119 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400e070 con 0x7f0b140cbab0 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b11d9b700 1 -- 172.21.15.100:0/3135030119 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0400bb60 con 0x7f0b140cbab0 2024-08-17T02:23:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b11d9b700 1 -- 172.21.15.100:0/3135030119 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04012430 con 0x7f0b140cbab0 2024-08-17T02:23:03.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- 172.21.15.100:0/3135030119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b140cbab0 msgr2=0x7f0b140cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 --2- 172.21.15.100:0/3135030119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b140cbab0 0x7f0b140cbf10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009e60 tx=0x7f0b04009780 comp rx=0 tx=0).stop 2024-08-17T02:23:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- 172.21.15.100:0/3135030119 shutdown_connections 2024-08-17T02:23:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 --2- 172.21.15.100:0/3135030119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b140cbab0 0x7f0b140cbf10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- 172.21.15.100:0/3135030119 >> 172.21.15.100:0/3135030119 conn(0x7f0b140cdee0 msgr2=0x7f0b140ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- 172.21.15.100:0/3135030119 shutdown_connections 2024-08-17T02:23:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- 172.21.15.100:0/3135030119 wait complete. 2024-08-17T02:23:03.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 Processor -- start 2024-08-17T02:23:03.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- start start 2024-08-17T02:23:03.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b140ad6e0 0x7f0b1412ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b140cbab0 0x7f0b14156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14140910 0x7f0b1412cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1412e770 con 0x7f0b140ad6e0 2024-08-17T02:23:03.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1412f140 con 0x7f0b14140910 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.447+0000 7f0b18887700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1412db60 con 0x7f0b140cbab0 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b140ad6e0 0x7f0b1412ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b140ad6e0 0x7f0b1412ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58934/0 (socket says 172.21.15.100:58934) 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 -- 172.21.15.100:0/2283935901 learned_addr learned my addr 172.21.15.100:0/2283935901 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b1359e700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14140910 0x7f0b1412cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b1259c700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b140cbab0 0x7f0b14156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 -- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b140cbab0 msgr2=0x7f0b14156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b140cbab0 0x7f0b14156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 -- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14140910 msgr2=0x7f0b1412cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14140910 0x7f0b1412cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 -- 172.21.15.100:0/2283935901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04009a20 con 0x7f0b140ad6e0 2024-08-17T02:23:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b1359e700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14140910 0x7f0b1412cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:23:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b12d9d700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b140ad6e0 0x7f0b1412ec00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00c4e0 tx=0x7f0afc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b0affd700 1 -- 172.21.15.100:0/2283935901 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc011590 con 0x7f0b140ad6e0 2024-08-17T02:23:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b0affd700 1 -- 172.21.15.100:0/2283935901 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc011710 con 0x7f0b140ad6e0 2024-08-17T02:23:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1412de40 con 0x7f0b140ad6e0 2024-08-17T02:23:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b0affd700 1 -- 172.21.15.100:0/2283935901 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc011ad0 con 0x7f0b140ad6e0 2024-08-17T02:23:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1412e400 con 0x7f0b140ad6e0 2024-08-17T02:23:03.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b140638f0 con 0x7f0b140ad6e0 2024-08-17T02:23:03.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.451+0000 7f0b0affd700 1 -- 172.21.15.100:0/2283935901 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0afc01b3b0 con 0x7f0b140ad6e0 2024-08-17T02:23:03.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.455+0000 7f0b0affd700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0af4060890 0x7f0af4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.455+0000 7f0b0affd700 1 -- 172.21.15.100:0/2283935901 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0afc07c0d0 con 0x7f0b140ad6e0 2024-08-17T02:23:03.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.455+0000 7f0b1259c700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0af4060890 0x7f0af4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.455+0000 7f0b1259c700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0af4060890 0x7f0af4062d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097c0 tx=0x7f0b04019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.459+0000 7f0b0affd700 1 -- 172.21.15.100:0/2283935901 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0afc04c520 con 0x7f0b140ad6e0 2024-08-17T02:23:03.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.619+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0b14068370 con 0x7f0b140ad6e0 2024-08-17T02:23:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.619+0000 7f0b0affd700 1 -- 172.21.15.100:0/2283935901 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0afc04fb40 con 0x7f0b140ad6e0 2024-08-17T02:23:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0af4060890 msgr2=0x7f0af4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0af4060890 0x7f0af4062d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097c0 tx=0x7f0b04019000 comp rx=0 tx=0).stop 2024-08-17T02:23:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b140ad6e0 msgr2=0x7f0b1412ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b140ad6e0 0x7f0b1412ec00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00c4e0 tx=0x7f0afc00c8a0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 shutdown_connections 2024-08-17T02:23:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0af4060890 0x7f0af4062d40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14140910 0x7f0b1412cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b140cbab0 0x7f0b14156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 --2- 172.21.15.100:0/2283935901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b140ad6e0 0x7f0b1412ec00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 >> 172.21.15.100:0/2283935901 conn(0x7f0b140cdee0 msgr2=0x7f0b140cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 shutdown_connections 2024-08-17T02:23:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.623+0000 7f0b18887700 1 -- 172.21.15.100:0/2283935901 wait complete. 2024-08-17T02:23:03.638 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000351990 2024-08-17T02:23:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d1359e700 1 Processor -- start 2024-08-17T02:23:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d1359e700 1 -- start start 2024-08-17T02:23:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d1359e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4d14142d40 0x7f4d140b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d1359e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d14140f70 con 0x7f4d140af190 2024-08-17T02:23:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d1359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d14141250 con 0x7f4d1406e470 2024-08-17T02:23:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d1359e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d14141530 con 0x7f4d14142d40 2024-08-17T02:23:03.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d11d9b700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4d140af190 0x7f4d140b0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45874/0 (socket says 172.21.15.100:45874) 2024-08-17T02:23:03.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d11d9b700 1 -- 172.21.15.100:0/2488579779 learned_addr learned my addr 172.21.15.100:0/2488579779 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:23:03.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.747+0000 7f4d12d9d700 1 --2- 172.21.15.100:0/2488579779 >> v2:172.21.15.100:3300/0 conn(0x7f4d14142d40 0x7f4d140b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1159a700 1 -- 172.21.15.100:0/2488579779 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450125565 0 0) 0x7f4d14140f70 con 0x7f4d140af190 2024-08-17T02:23:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d12d9d700 1 -- 172.21.15.100:0/2488579779 >> v1:172.21.15.123:6789/0 conn(0x7f4d1406e470 legacy=0x7f4d140aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d12d9d700 1 -- 172.21.15.100:0/2488579779 >> v1:172.21.15.100:6789/0 conn(0x7f4d140af190 legacy=0x7f4d140b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d12d9d700 1 -- 172.21.15.100:0/2488579779 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14141810 con 0x7f4d14142d40 2024-08-17T02:23:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d12d9d700 1 --2- 172.21.15.100:0/2488579779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d14142d40 0x7f4d140b1600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009e60 tx=0x7f4d0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cbcdaccf2c10a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1159a700 1 -- 172.21.15.100:0/2488579779 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00e070 con 0x7f4d14142d40 2024-08-17T02:23:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1159a700 1 -- 172.21.15.100:0/2488579779 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c00bc30 con 0x7f4d14142d40 2024-08-17T02:23:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1159a700 1 -- 172.21.15.100:0/2488579779 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c012430 con 0x7f4d14142d40 2024-08-17T02:23:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 -- 172.21.15.100:0/2488579779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d14142d40 msgr2=0x7f4d140b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 --2- 172.21.15.100:0/2488579779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d14142d40 0x7f4d140b1600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009e60 tx=0x7f4d0c009780 comp rx=0 tx=0).stop 2024-08-17T02:23:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 -- 172.21.15.100:0/2488579779 shutdown_connections 2024-08-17T02:23:03.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 --2- 172.21.15.100:0/2488579779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d14142d40 0x7f4d140b1600 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 -- 172.21.15.100:0/2488579779 >> 172.21.15.100:0/2488579779 conn(0x7f4d140678a0 msgr2=0x7f4d14067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:03.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 -- 172.21.15.100:0/2488579779 shutdown_connections 2024-08-17T02:23:03.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 -- 172.21.15.100:0/2488579779 wait complete. 2024-08-17T02:23:03.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 Processor -- start 2024-08-17T02:23:03.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.751+0000 7f4d1359e700 1 -- start start 2024-08-17T02:23:03.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d1406e470 0x7f4d140c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1359e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af190 0x7f4d1415ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d14142d40 0x7f4d140ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1359e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140c7960 con 0x7f4d140af190 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140c9e80 con 0x7f4d14142d40 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140cafb0 con 0x7f4d1406e470 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af190 0x7f4d1415ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af190 0x7f4d1415ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58964/0 (socket says 172.21.15.100:58964) 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 -- 172.21.15.100:0/302505441 learned_addr learned my addr 172.21.15.100:0/302505441 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d12d9d700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d14142d40 0x7f4d140ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1259c700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d1406e470 0x7f4d140c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 -- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d1406e470 msgr2=0x7f4d140c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d1406e470 0x7f4d140c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 -- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d14142d40 msgr2=0x7f4d140ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d14142d40 0x7f4d140ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 -- 172.21.15.100:0/302505441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c009a20 con 0x7f4d140af190 2024-08-17T02:23:03.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d11d9b700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af190 0x7f4d1415ce80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00c610 tx=0x7f4cfc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d0a7fc700 1 -- 172.21.15.100:0/302505441 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc005b30 con 0x7f4d140af190 2024-08-17T02:23:03.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d0a7fc700 1 -- 172.21.15.100:0/302505441 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc005c90 con 0x7f4d140af190 2024-08-17T02:23:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d140c0fa0 con 0x7f4d140af190 2024-08-17T02:23:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d0a7fc700 1 -- 172.21.15.100:0/302505441 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc007300 con 0x7f4d140af190 2024-08-17T02:23:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.755+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d140c9830 con 0x7f4d140af190 2024-08-17T02:23:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.759+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d14064330 con 0x7f4d140af190 2024-08-17T02:23:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.759+0000 7f4d0a7fc700 1 -- 172.21.15.100:0/302505441 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4cfc01b3b0 con 0x7f4d140af190 2024-08-17T02:23:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.759+0000 7f4d0a7fc700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4cf8060860 0x7f4cf8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.759+0000 7f4d0a7fc700 1 -- 172.21.15.100:0/302505441 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4cfc07c060 con 0x7f4d140af190 2024-08-17T02:23:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.759+0000 7f4d1259c700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4cf8060860 0x7f4cf8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:03.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.759+0000 7f4d1259c700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4cf8060860 0x7f4cf8062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d04007ea0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.763+0000 7f4d0a7fc700 1 -- 172.21.15.100:0/302505441 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4cfc04c410 con 0x7f4d140af190 2024-08-17T02:23:03.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:03.951+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000351990", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d140c9fc0 con 0x7f4cf8060860 2024-08-17T02:23:06.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.075+0000 7f4d0a7fc700 1 -- 172.21.15.100:0/302505441 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f4d140c9fc0 con 0x7f4cf8060860 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4cf8060860 msgr2=0x7f4cf8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4cf8060860 0x7f4cf8062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d04007ea0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af190 msgr2=0x7f4d1415ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af190 0x7f4d1415ce80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00c610 tx=0x7f4cfc000fe0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 shutdown_connections 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4cf8060860 0x7f4cf8062d10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d14142d40 0x7f4d140ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af190 0x7f4d1415ce80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 --2- 172.21.15.100:0/302505441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d1406e470 0x7f4d140c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 >> 172.21.15.100:0/302505441 conn(0x7f4d140678a0 msgr2=0x7f4d1415dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 shutdown_connections 2024-08-17T02:23:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.079+0000 7f4d1359e700 1 -- 172.21.15.100:0/302505441 wait complete. 2024-08-17T02:23:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:Volume created successfully (no MDS daemons created) 2024-08-17T02:23:06.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-17T02:23:06.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.195+0000 7fd7fb852700 1 Processor -- start 2024-08-17T02:23:06.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.195+0000 7fd7fb852700 1 -- start start 2024-08-17T02:23:06.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7f40688d0 0x7fd7f4068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd7f406e890 0x7fd7f40ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 0x7fd7f415a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f40af070 con 0x7fd7f40688d0 2024-08-17T02:23:06.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f40abca0 con 0x7fd7f406e890 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f40cac60 con 0x7fd7f415a270 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7f40688d0 0x7fd7f4068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7f40688d0 0x7fd7f4068ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49152/0 (socket says 172.21.15.100:49152) 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 -- 172.21.15.100:0/320419400 learned_addr learned my addr 172.21.15.100:0/320419400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f8ded700 1 --2- 172.21.15.100:0/320419400 >> v2:172.21.15.123:3300/0 conn(0x7fd7f406e890 0x7fd7f40ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f9def700 1 --2- 172.21.15.100:0/320419400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 0x7fd7f415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 -- 172.21.15.100:0/320419400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 msgr2=0x7fd7f415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 --2- 172.21.15.100:0/320419400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 0x7fd7f415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 -- 172.21.15.100:0/320419400 >> v2:172.21.15.123:3300/0 conn(0x7fd7f406e890 msgr2=0x7fd7f40ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 --2- 172.21.15.100:0/320419400 >> v2:172.21.15.123:3300/0 conn(0x7fd7f406e890 0x7fd7f40ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 -- 172.21.15.100:0/320419400 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40abed0 con 0x7fd7f40688d0 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f8ded700 1 --2- 172.21.15.100:0/320419400 >> v2:172.21.15.123:3300/0 conn(0x7fd7f406e890 0x7fd7f40ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:23:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7f95ee700 1 --2- 172.21.15.100:0/320419400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f4068ca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4064150 tx=0x7fd7f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5935d51a559d8b44 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7ebfff700 1 -- 172.21.15.100:0/320419400 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000b860 con 0x7fd7f40688d0 2024-08-17T02:23:06.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7ebfff700 1 -- 172.21.15.100:0/320419400 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f000b9c0 con 0x7fd7f40688d0 2024-08-17T02:23:06.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7ebfff700 1 -- 172.21.15.100:0/320419400 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000bd60 con 0x7fd7f40688d0 2024-08-17T02:23:06.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 -- 172.21.15.100:0/320419400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 msgr2=0x7fd7f4068ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 --2- 172.21.15.100:0/320419400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f4068ca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4064150 tx=0x7fd7f0009250 comp rx=0 tx=0).stop 2024-08-17T02:23:06.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 -- 172.21.15.100:0/320419400 shutdown_connections 2024-08-17T02:23:06.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 --2- 172.21.15.100:0/320419400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 0x7fd7f415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 --2- 172.21.15.100:0/320419400 >> v2:172.21.15.123:3300/0 conn(0x7fd7f406e890 0x7fd7f40ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 --2- 172.21.15.100:0/320419400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f4068ca0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 -- 172.21.15.100:0/320419400 >> 172.21.15.100:0/320419400 conn(0x7fd7f40cd160 msgr2=0x7fd7f406d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:06.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 -- 172.21.15.100:0/320419400 shutdown_connections 2024-08-17T02:23:06.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.199+0000 7fd7fb852700 1 -- 172.21.15.100:0/320419400 wait complete. 2024-08-17T02:23:06.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7fb852700 1 Processor -- start 2024-08-17T02:23:06.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7fb852700 1 -- start start 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7fb852700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f416ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7fb852700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f406e890 0x7fd7f416b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7fb852700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 0x7fd7f41569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7fb852700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f4158380 con 0x7fd7f40688d0 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7fb852700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f4157770 con 0x7fd7f406e890 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7fb852700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f4157a50 con 0x7fd7f415a270 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f416ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f416ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49156/0 (socket says 172.21.15.100:49156) 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 -- 172.21.15.100:0/3912487902 learned_addr learned my addr 172.21.15.100:0/3912487902 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:06.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f8ded700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f406e890 0x7fd7f416b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f9def700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 0x7fd7f41569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 -- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 msgr2=0x7fd7f41569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 0x7fd7f41569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 -- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f406e890 msgr2=0x7fd7f416b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f406e890 0x7fd7f416b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 -- 172.21.15.100:0/3912487902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00094f0 con 0x7fd7f40688d0 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7f95ee700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f416ab40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009290 tx=0x7fd7f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.203+0000 7fd7e97fa700 1 -- 172.21.15.100:0/3912487902 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0011e40 con 0x7fd7f40688d0 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7e97fa700 1 -- 172.21.15.100:0/3912487902 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f001f070 con 0x7fd7f40688d0 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f4157d30 con 0x7fd7f40688d0 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7e97fa700 1 -- 172.21.15.100:0/3912487902 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00266d0 con 0x7fd7f40688d0 2024-08-17T02:23:06.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f413ece0 con 0x7fd7f40688d0 2024-08-17T02:23:06.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f40645c0 con 0x7fd7f40688d0 2024-08-17T02:23:06.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7e97fa700 1 -- 172.21.15.100:0/3912487902 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd7f0026a40 con 0x7fd7f40688d0 2024-08-17T02:23:06.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7e97fa700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd7d8060860 0x7fd7d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7e97fa700 1 -- 172.21.15.100:0/3912487902 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fd7f0021070 con 0x7fd7f40688d0 2024-08-17T02:23:06.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.207+0000 7fd7f8ded700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd7d8060860 0x7fd7d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.211+0000 7fd7f8ded700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd7d8060860 0x7fd7d8062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd7f415bd20 tx=0x7fd7e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.215+0000 7fd7e97fa700 1 -- 172.21.15.100:0/3912487902 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd7f004ed60 con 0x7fd7f40688d0 2024-08-17T02:23:06.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.399+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7f4067270 con 0x7fd7d8060860 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7e97fa700 1 -- 172.21.15.100:0/3912487902 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7fd7f4067270 con 0x7fd7d8060860 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volume_0000000000351990" 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-08-17T02:23:06.405 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-08-17T02:23:06.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd7d8060860 msgr2=0x7fd7d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd7d8060860 0x7fd7d8062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd7f415bd20 tx=0x7fd7e4008040 comp rx=0 tx=0).stop 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 msgr2=0x7fd7f416ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f416ab40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009290 tx=0x7fd7f0017000 comp rx=0 tx=0).stop 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 shutdown_connections 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd7d8060860 0x7fd7d8062d10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7f415a270 0x7fd7f41569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f406e890 0x7fd7f416b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 --2- 172.21.15.100:0/3912487902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f40688d0 0x7fd7f416ab40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 >> 172.21.15.100:0/3912487902 conn(0x7fd7f40cd160 msgr2=0x7fd7f40ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:06.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 shutdown_connections 2024-08-17T02:23:06.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.403+0000 7fd7fb852700 1 -- 172.21.15.100:0/3912487902 wait complete. 2024-08-17T02:23:06.419 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2024-08-17T02:23:06.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f831cc25700 1 Processor -- start 2024-08-17T02:23:06.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f831cc25700 1 -- start start 2024-08-17T02:23:06.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f831cc25700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83181590b0 0x7f8318066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f831cc25700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 0x7f8318067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f831cc25700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83180ce210 con 0x7f83180cd2b0 2024-08-17T02:23:06.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f831cc25700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83180cba00 con 0x7f83181590b0 2024-08-17T02:23:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f831cc25700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83180cdc90 con 0x7f8318159480 2024-08-17T02:23:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f83177fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83181590b0 0x7f8318066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f8317fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f83180cd2b0 0x7f831815ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:38808/0 (socket says 172.21.15.100:38808) 2024-08-17T02:23:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f8317fff700 1 -- 172.21.15.100:0/1228425320 learned_addr learned my addr 172.21.15.100:0/1228425320 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:23:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.531+0000 7f8316ffd700 1 --2- 172.21.15.100:0/1228425320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 0x7f8318067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83177fe700 1 -- 172.21.15.100:0/1228425320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 msgr2=0x7f8318067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83177fe700 1 --2- 172.21.15.100:0/1228425320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 0x7f8318067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83177fe700 1 -- 172.21.15.100:0/1228425320 >> v1:172.21.15.100:6789/0 conn(0x7f83180cd2b0 legacy=0x7f831815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83167fc700 1 -- 172.21.15.100:0/1228425320 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552758696 0 0) 0x7f83180ce210 con 0x7f83180cd2b0 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83177fe700 1 -- 172.21.15.100:0/1228425320 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180cb440 con 0x7f83181590b0 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83177fe700 1 --2- 172.21.15.100:0/1228425320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f8318066e00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098e0 tx=0x7f830c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc51094de757565 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83167fc700 1 -- 172.21.15.100:0/1228425320 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c00b640 con 0x7f83181590b0 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83167fc700 1 -- 172.21.15.100:0/1228425320 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c00b7a0 con 0x7f83181590b0 2024-08-17T02:23:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f83167fc700 1 -- 172.21.15.100:0/1228425320 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c00bb40 con 0x7f83181590b0 2024-08-17T02:23:06.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f831cc25700 1 -- 172.21.15.100:0/1228425320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 msgr2=0x7f8318066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f831cc25700 1 --2- 172.21.15.100:0/1228425320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f8318066e00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098e0 tx=0x7f830c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f831cc25700 1 -- 172.21.15.100:0/1228425320 shutdown_connections 2024-08-17T02:23:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f831cc25700 1 --2- 172.21.15.100:0/1228425320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 0x7f8318067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f831cc25700 1 --2- 172.21.15.100:0/1228425320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f8318066e00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f831cc25700 1 -- 172.21.15.100:0/1228425320 >> 172.21.15.100:0/1228425320 conn(0x7f83180d0b90 msgr2=0x7f83180ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f831cc25700 1 -- 172.21.15.100:0/1228425320 shutdown_connections 2024-08-17T02:23:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.535+0000 7f831cc25700 1 -- 172.21.15.100:0/1228425320 wait complete. 2024-08-17T02:23:06.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f831cc25700 1 Processor -- start 2024-08-17T02:23:06.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f831cc25700 1 -- start start 2024-08-17T02:23:06.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f831cc25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83180cd2b0 0x7f83181698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f831cc25700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f83180c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f831cc25700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 0x7f83180b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f831cc25700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83181565a0 con 0x7f83181590b0 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f831cc25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831815e660 con 0x7f83180cd2b0 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f831cc25700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83180ca070 con 0x7f8318159480 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f83180c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f83180c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49174/0 (socket says 172.21.15.100:49174) 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f8316ffd700 1 -- 172.21.15.100:0/2528672448 learned_addr learned my addr 172.21.15.100:0/2528672448 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f83177fe700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83180cd2b0 0x7f83181698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.539+0000 7f8317fff700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 0x7f83180b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f8316ffd700 1 -- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 msgr2=0x7f83180b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f8316ffd700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 0x7f83180b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f8316ffd700 1 -- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83180cd2b0 msgr2=0x7f83181698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f8316ffd700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83180cd2b0 0x7f83181698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f8316ffd700 1 -- 172.21.15.100:0/2528672448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c009420 con 0x7f83181590b0 2024-08-17T02:23:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f8316ffd700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f83180c7900 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f830800e930 tx=0x7f83080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f82ff7fe700 1 -- 172.21.15.100:0/2528672448 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308013680 con 0x7f83181590b0 2024-08-17T02:23:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f82ff7fe700 1 -- 172.21.15.100:0/2528672448 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83080137e0 con 0x7f83181590b0 2024-08-17T02:23:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83180ca350 con 0x7f83181590b0 2024-08-17T02:23:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f82ff7fe700 1 -- 172.21.15.100:0/2528672448 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308013ba0 con 0x7f83181590b0 2024-08-17T02:23:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831815c8a0 con 0x7f83181590b0 2024-08-17T02:23:06.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8318064330 con 0x7f83181590b0 2024-08-17T02:23:06.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f82ff7fe700 1 -- 172.21.15.100:0/2528672448 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f830800c3e0 con 0x7f83181590b0 2024-08-17T02:23:06.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f82ff7fe700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8304060860 0x7f8304062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f82ff7fe700 1 -- 172.21.15.100:0/2528672448 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f830807cf10 con 0x7f83181590b0 2024-08-17T02:23:06.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.543+0000 7f83177fe700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8304060860 0x7f8304062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.547+0000 7f83177fe700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8304060860 0x7f8304062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f830c0093f0 tx=0x7f830c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.551+0000 7f82ff7fe700 1 -- 172.21.15.100:0/2528672448 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f830804cfe0 con 0x7f83181590b0 2024-08-17T02:23:06.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.715+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f8318070cf0 con 0x7f83181590b0 2024-08-17T02:23:06.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f82ff7fe700 1 -- 172.21.15.100:0/2528672448 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v36) v1 ==== 97+0+8050 (secure 0 0 0) 0x7f8308050600 con 0x7f83181590b0 2024-08-17T02:23:06.721 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:23:06.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8304060860 msgr2=0x7f8304062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8304060860 0x7f8304062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f830c0093f0 tx=0x7f830c017000 comp rx=0 tx=0).stop 2024-08-17T02:23:06.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 msgr2=0x7f83180c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f83180c7900 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f830800e930 tx=0x7f83080091a0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 shutdown_connections 2024-08-17T02:23:06.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8304060860 0x7f8304062d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8318159480 0x7f83180b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83181590b0 0x7f83180c7900 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 --2- 172.21.15.100:0/2528672448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83180cd2b0 0x7f83181698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 >> 172.21.15.100:0/2528672448 conn(0x7f83180d0b90 msgr2=0x7f83180c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:06.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 shutdown_connections 2024-08-17T02:23:06.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.719+0000 7f831cc25700 1 -- 172.21.15.100:0/2528672448 wait complete. 2024-08-17T02:23:06.736 INFO:teuthology.orchestra.run.smithi100.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:50.544046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:51.275880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs.volume_0000000000351990.meta","create_time":"2024-08-17T02:23:03.999505+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000351990"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000351990.data","create_time":"2024-08-17T02:23:04.098167+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000351990"}}}] 2024-08-17T02:23:06.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000351990 --format=json 2024-08-17T02:23:06.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.835+0000 7fea8359e700 1 Processor -- start 2024-08-17T02:23:06.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.835+0000 7fea8359e700 1 -- start start 2024-08-17T02:23:06.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fea840ab540 0x7fea8406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea84155720 0x7fea840b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea840aba70 con 0x7fea8406e9d0 2024-08-17T02:23:06.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea840abd50 con 0x7fea840ab540 2024-08-17T02:23:06.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea840ac030 con 0x7fea84155720 2024-08-17T02:23:06.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea81d9b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fea840ab540 0x7fea8406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea81d9b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fea840ab540 0x7fea8406eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49186/0 (socket says 172.21.15.100:49186) 2024-08-17T02:23:06.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea81d9b700 1 -- 172.21.15.100:0/4172818884 learned_addr learned my addr 172.21.15.100:0/4172818884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:06.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea82d9d700 1 --2- 172.21.15.100:0/4172818884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea84155720 0x7fea840b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea81d9b700 1 -- 172.21.15.100:0/4172818884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea84155720 msgr2=0x7fea840b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea81d9b700 1 --2- 172.21.15.100:0/4172818884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea84155720 0x7fea840b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea81d9b700 1 -- 172.21.15.100:0/4172818884 >> v1:172.21.15.123:6789/0 conn(0x7fea8406e9d0 legacy=0x7fea84155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea81d9b700 1 -- 172.21.15.100:0/4172818884 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840ac310 con 0x7fea840ab540 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea81d9b700 1 --2- 172.21.15.100:0/4172818884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840ab540 0x7fea8406eda0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009f20 tx=0x7fea6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384e089834f8982 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8159a700 1 -- 172.21.15.100:0/4172818884 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c00e070 con 0x7fea840ab540 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8159a700 1 -- 172.21.15.100:0/4172818884 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c00b9d0 con 0x7fea840ab540 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8159a700 1 -- 172.21.15.100:0/4172818884 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0092e0 con 0x7fea840ab540 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 -- 172.21.15.100:0/4172818884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840ab540 msgr2=0x7fea8406eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 --2- 172.21.15.100:0/4172818884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840ab540 0x7fea8406eda0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009f20 tx=0x7fea6c009840 comp rx=0 tx=0).stop 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 -- 172.21.15.100:0/4172818884 shutdown_connections 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 --2- 172.21.15.100:0/4172818884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea84155720 0x7fea840b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 --2- 172.21.15.100:0/4172818884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840ab540 0x7fea8406eda0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 -- 172.21.15.100:0/4172818884 >> 172.21.15.100:0/4172818884 conn(0x7fea840678d0 msgr2=0x7fea84067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:06.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 -- 172.21.15.100:0/4172818884 shutdown_connections 2024-08-17T02:23:06.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.839+0000 7fea8359e700 1 -- 172.21.15.100:0/4172818884 wait complete. 2024-08-17T02:23:06.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 Processor -- start 2024-08-17T02:23:06.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 -- start start 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea8406e9d0 0x7fea841371e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea840ab540 0x7fea8414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea84155720 0x7fea8414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea841509a0 con 0x7fea84155720 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8414ae00 con 0x7fea840ab540 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea841530f0 con 0x7fea8406e9d0 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea84155720 0x7fea8414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea84155720 0x7fea8414fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49200/0 (socket says 172.21.15.100:49200) 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 -- 172.21.15.100:0/2087910625 learned_addr learned my addr 172.21.15.100:0/2087910625 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8259c700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea8406e9d0 0x7fea841371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea81d9b700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea840ab540 0x7fea8414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 -- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea8406e9d0 msgr2=0x7fea841371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea8406e9d0 0x7fea841371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 -- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea840ab540 msgr2=0x7fea8414f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea840ab540 0x7fea8414f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 -- 172.21.15.100:0/2087910625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c009ae0 con 0x7fea84155720 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea81d9b700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea840ab540 0x7fea8414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea82d9d700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea84155720 0x7fea8414fdf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea7400e8d0 tx=0x7fea740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea7a7fc700 1 -- 172.21.15.100:0/2087910625 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74013680 con 0x7fea84155720 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea7a7fc700 1 -- 172.21.15.100:0/2087910625 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740137e0 con 0x7fea84155720 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea841503e0 con 0x7fea84155720 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea7a7fc700 1 -- 172.21.15.100:0/2087910625 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74013ba0 con 0x7fea84155720 2024-08-17T02:23:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.843+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea84152d80 con 0x7fea84155720 2024-08-17T02:23:06.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.847+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea84152be0 con 0x7fea84155720 2024-08-17T02:23:06.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.847+0000 7fea7a7fc700 1 -- 172.21.15.100:0/2087910625 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fea740123d0 con 0x7fea84155720 2024-08-17T02:23:06.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.847+0000 7fea7a7fc700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fea68060860 0x7fea68062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:06.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.847+0000 7fea7a7fc700 1 -- 172.21.15.100:0/2087910625 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fea7407d100 con 0x7fea84155720 2024-08-17T02:23:06.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.847+0000 7fea8259c700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fea68060860 0x7fea68062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.847+0000 7fea8259c700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fea68060860 0x7fea68062d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea8416cc00 tx=0x7fea7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:06.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:06.851+0000 7fea7a7fc700 1 -- 172.21.15.100:0/2087910625 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fea84152be0 con 0x7fea84155720 2024-08-17T02:23:07.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000351990", "format": "json"} v 0) v1 -- 0x7fea84152be0 con 0x7fea84155720 2024-08-17T02:23:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea7a7fc700 1 -- 172.21.15.100:0/2087910625 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000351990", "format": "json"}]=0 v16) v1 ==== 110+0+1847 (secure 0 0 0) 0x7fea84152be0 con 0x7fea84155720 2024-08-17T02:23:07.054 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:23:07.054 INFO:teuthology.orchestra.run.smithi100.stdout:{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:23:05.071644+0000","modified":"2024-08-17T02:23:06.938110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000351990","balancer":"","standby_count_wanted":1},"id":3} 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fea68060860 msgr2=0x7fea68062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fea68060860 0x7fea68062d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea8416cc00 tx=0x7fea7c009040 comp rx=0 tx=0).stop 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea84155720 msgr2=0x7fea8414fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea84155720 0x7fea8414fdf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea7400e8d0 tx=0x7fea740091a0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 shutdown_connections 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fea68060860 0x7fea68062d10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea84155720 0x7fea8414fdf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea840ab540 0x7fea8414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 --2- 172.21.15.100:0/2087910625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea8406e9d0 0x7fea841371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.051+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 >> 172.21.15.100:0/2087910625 conn(0x7fea840678d0 msgr2=0x7fea840acd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.055+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 shutdown_connections 2024-08-17T02:23:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.055+0000 7fea8359e700 1 -- 172.21.15.100:0/2087910625 wait complete. 2024-08-17T02:23:07.077 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000351990 --yes-i-really-mean-it 2024-08-17T02:23:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe204e1b700 1 Processor -- start 2024-08-17T02:23:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe204e1b700 1 -- start start 2024-08-17T02:23:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe204e1b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe20015d540 0x7fe20015d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe204e1b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe204e1b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000cd4e0 con 0x7fe2000c8b20 2024-08-17T02:23:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe204e1b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000cddc0 con 0x7fe20015d540 2024-08-17T02:23:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe204e1b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000cc8d0 con 0x7fe2000cd970 2024-08-17T02:23:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fdd9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fed9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe2000c8b20 0x7fe2000ad9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:33400/0 (socket says 172.21.15.100:33400) 2024-08-17T02:23:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fdd9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:44936/0 (socket says 172.21.15.100:44936) 2024-08-17T02:23:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fed9d700 1 -- 172.21.15.100:0/2855508577 learned_addr learned my addr 172.21.15.100:0/2855508577 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:23:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fe59c700 1 --2- 172.21.15.100:0/2855508577 >> v2:172.21.15.123:3300/0 conn(0x7fe20015d540 0x7fe20015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:07.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fd59a700 1 -- 172.21.15.100:0/2855508577 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238713413 0 0) 0x7fe2000cd4e0 con 0x7fe2000c8b20 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fd59a700 1 -- 172.21.15.100:0/2855508577 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8003570 con 0x7fe2000c8b20 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fdd9b700 1 -- 172.21.15.100:0/2855508577 >> v2:172.21.15.123:3300/0 conn(0x7fe20015d540 msgr2=0x7fe20015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fdd9b700 1 --2- 172.21.15.100:0/2855508577 >> v2:172.21.15.123:3300/0 conn(0x7fe20015d540 0x7fe20015d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fdd9b700 1 -- 172.21.15.100:0/2855508577 >> v1:172.21.15.123:6789/0 conn(0x7fe2000c8b20 legacy=0x7fe2000ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fdd9b700 1 -- 172.21.15.100:0/2855508577 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000ccbb0 con 0x7fe2000cd970 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fe59c700 1 --2- 172.21.15.100:0/2855508577 >> v2:172.21.15.123:3300/0 conn(0x7fe20015d540 0x7fe20015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fdd9b700 1 --2- 172.21.15.100:0/2855508577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c85e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00098e0 tx=0x7fe1f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1c2b8ea8729c27 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fd59a700 1 -- 172.21.15.100:0/2855508577 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0011630 con 0x7fe2000cd970 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.175+0000 7fe1fd59a700 1 -- 172.21.15.100:0/2855508577 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f0011790 con 0x7fe2000cd970 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe1fd59a700 1 -- 172.21.15.100:0/2855508577 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0011b30 con 0x7fe2000cd970 2024-08-17T02:23:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 -- 172.21.15.100:0/2855508577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 msgr2=0x7fe2000c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:07.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 --2- 172.21.15.100:0/2855508577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c85e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00098e0 tx=0x7fe1f00091a0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 -- 172.21.15.100:0/2855508577 shutdown_connections 2024-08-17T02:23:07.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 --2- 172.21.15.100:0/2855508577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c85e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 --2- 172.21.15.100:0/2855508577 >> v2:172.21.15.123:3300/0 conn(0x7fe20015d540 0x7fe20015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 -- 172.21.15.100:0/2855508577 >> 172.21.15.100:0/2855508577 conn(0x7fe2000caab0 msgr2=0x7fe2000caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:23:07.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 -- 172.21.15.100:0/2855508577 shutdown_connections 2024-08-17T02:23:07.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 -- 172.21.15.100:0/2855508577 wait complete. 2024-08-17T02:23:07.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 Processor -- start 2024-08-17T02:23:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.179+0000 7fe204e1b700 1 -- start start 2024-08-17T02:23:07.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2000c8b20 0x7fe20015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:07.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20015d540 0x7fe20015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20015f780 con 0x7fe2000c8b20 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c3510 con 0x7fe20015d540 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20015ee50 con 0x7fe2000cd970 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2000c8b20 0x7fe20015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2000c8b20 0x7fe20015b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49212/0 (socket says 172.21.15.100:49212) 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20015d540 0x7fe20015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 -- 172.21.15.100:0/92572233 learned_addr learned my addr 172.21.15.100:0/92572233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fdd9b700 1 --2- 172.21.15.100:0/92572233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 -- 172.21.15.100:0/92572233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 msgr2=0x7fe2000c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 --2- 172.21.15.100:0/92572233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2000cd970 0x7fe2000c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 -- 172.21.15.100:0/92572233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20015d540 msgr2=0x7fe20015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:23:07.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 --2- 172.21.15.100:0/92572233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20015d540 0x7fe20015fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:23:07.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 -- 172.21.15.100:0/92572233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f4009420 con 0x7fe2000c8b20 2024-08-17T02:23:07.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1fe59c700 1 --2- 172.21.15.100:0/92572233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2000c8b20 0x7fe20015b730 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400ec80 tx=0x7fe1f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:07.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1ee7fc700 1 -- 172.21.15.100:0/92572233 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f400c850 con 0x7fe2000c8b20 2024-08-17T02:23:07.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1ee7fc700 1 -- 172.21.15.100:0/92572233 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f400c9b0 con 0x7fe2000c8b20 2024-08-17T02:23:07.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe1ee7fc700 1 -- 172.21.15.100:0/92572233 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f400cd30 con 0x7fe2000c8b20 2024-08-17T02:23:07.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 -- 172.21.15.100:0/92572233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f0009420 con 0x7fe2000c8b20 2024-08-17T02:23:07.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 -- 172.21.15.100:0/92572233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20015f410 con 0x7fe2000c8b20 2024-08-17T02:23:07.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.183+0000 7fe204e1b700 1 -- 172.21.15.100:0/92572233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2000645c0 con 0x7fe2000c8b20 2024-08-17T02:23:07.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.187+0000 7fe1ee7fc700 1 -- 172.21.15.100:0/92572233 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe1f4013680 con 0x7fe2000c8b20 2024-08-17T02:23:07.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.187+0000 7fe1ee7fc700 1 --2- 172.21.15.100:0/92572233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1e0060860 0x7fe1e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:23:07.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.187+0000 7fe1ee7fc700 1 -- 172.21.15.100:0/92572233 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fe1f407d4e0 con 0x7fe2000c8b20 2024-08-17T02:23:07.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.187+0000 7fe1fdd9b700 1 --2- 172.21.15.100:0/92572233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1e0060860 0x7fe1e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:23:07.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.187+0000 7fe1fdd9b700 1 --2- 172.21.15.100:0/92572233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1e0060860 0x7fe1e0062d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00098e0 tx=0x7fe1f0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:23:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.191+0000 7fe1ee7fc700 1 -- 172.21.15.100:0/92572233 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe1f4051040 con 0x7fe2000c8b20 2024-08-17T02:23:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:23:07.383+0000 7fe204e1b700 1 -- 172.21.15.100:0/92572233 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000351990", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fe20015f8c0 con 0x7fe1e0060860 2024-08-17T02:23:07.643 INFO:tasks.ceph.mds.c.smithi100.stderr:2024-08-17T02:23:07.639+0000 7fa7583d3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:23:08.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:23:08.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:23:38.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:23:38.824 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:24:08.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:24:08.921 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:24:38.972 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:24:38.975 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:07.091 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-08-17T02:25:07.093 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-17T02:25:07.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-17T02:25:07.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-17T02:25:07.132 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-17T02:25:07.132 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-08-17T02:25:07.132 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:25:07.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-17T02:25:07.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[32676]: fuse finished with error 0 and tester_r 0 2024-08-17T02:25:07.167 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:25:07.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:07.167 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-17T02:25:07.264 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-08-17T02:25:07.265 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:25:07.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-08-17T02:25:07.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[29612]: fuse finished with error 0 and tester_r 0 2024-08-17T02:25:07.282 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:25:07.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:07.283 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-17T02:25:07.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:25:07.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:07.443 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-17T02:25:07.450 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-08-17T02:25:07.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:25:07.451 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:25:07.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:07.451 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-17T02:25:07.457 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-08-17T02:25:07.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:25:07.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:25:07.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0d4acb700 1 Processor -- start 2024-08-17T02:25:07.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0d4acb700 1 -- start start 2024-08-17T02:25:07.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0d4acb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d0159040 0x7fe0d00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0d4acb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe0d015d5b0 0x7fe0d00d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0d4acb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d00cf190 con 0x7fe0d00d0010 2024-08-17T02:25:07.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0d4acb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00d09a0 con 0x7fe0d015d5b0 2024-08-17T02:25:07.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0d4acb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00ce5e0 con 0x7fe0d0159040 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0cf7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d0159040 0x7fe0d00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.600+0000 7fe0cffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe0d015d5b0 0x7fe0d00d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0ceffd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe0d00d0010 0x7fe0d015cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:46262/0 (socket says 172.21.15.100:46262) 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0cf7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d0159040 0x7fe0d00cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:43214/0 (socket says 172.21.15.100:43214) 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0ceffd700 1 -- 172.21.15.100:0/2648355725 learned_addr learned my addr 172.21.15.100:0/2648355725 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0cffff700 1 -- 172.21.15.100:0/2648355725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d0159040 msgr2=0x7fe0d00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0cffff700 1 --2- 172.21.15.100:0/2648355725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d0159040 0x7fe0d00cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0cffff700 1 -- 172.21.15.100:0/2648355725 >> v1:172.21.15.123:6789/0 conn(0x7fe0d00d0010 legacy=0x7fe0d015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0ce7fc700 1 -- 172.21.15.100:0/2648355725 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670892153 0 0) 0x7fe0d00cf190 con 0x7fe0d00d0010 2024-08-17T02:25:07.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0cffff700 1 -- 172.21.15.100:0/2648355725 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00ce860 con 0x7fe0d015d5b0 2024-08-17T02:25:07.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0cf7fe700 1 --2- 172.21.15.100:0/2648355725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d0159040 0x7fe0d00cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:07.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0cffff700 1 --2- 172.21.15.100:0/2648355725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 0x7fe0d00d0460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009e60 tx=0x7fe0c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a85e1562d03c9f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:07.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0ce7fc700 1 -- 172.21.15.100:0/2648355725 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800e070 con 0x7fe0d015d5b0 2024-08-17T02:25:07.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0ce7fc700 1 -- 172.21.15.100:0/2648355725 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c800bb60 con 0x7fe0d015d5b0 2024-08-17T02:25:07.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0ce7fc700 1 -- 172.21.15.100:0/2648355725 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8012430 con 0x7fe0d015d5b0 2024-08-17T02:25:07.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0d4acb700 1 -- 172.21.15.100:0/2648355725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 msgr2=0x7fe0d00d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/2648355725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 0x7fe0d00d0460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009e60 tx=0x7fe0c8009780 comp rx=0 tx=0).stop 2024-08-17T02:25:07.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0d4acb700 1 -- 172.21.15.100:0/2648355725 shutdown_connections 2024-08-17T02:25:07.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/2648355725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 0x7fe0d00d0460 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/2648355725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d0159040 0x7fe0d00cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0d4acb700 1 -- 172.21.15.100:0/2648355725 >> 172.21.15.100:0/2648355725 conn(0x7fe0d0068250 msgr2=0x7fe0d0068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:07.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0d4acb700 1 -- 172.21.15.100:0/2648355725 shutdown_connections 2024-08-17T02:25:07.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.604+0000 7fe0d4acb700 1 -- 172.21.15.100:0/2648355725 wait complete. 2024-08-17T02:25:07.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0d4acb700 1 Processor -- start 2024-08-17T02:25:07.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0d4acb700 1 -- start start 2024-08-17T02:25:07.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0d4acb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d00d0010 0x7fe0d01693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0d4acb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d0159040 0x7fe0d015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0d4acb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 0x7fe0d0152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0d4acb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00afef0 con 0x7fe0d0159040 2024-08-17T02:25:07.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0d4acb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00b01d0 con 0x7fe0d015d5b0 2024-08-17T02:25:07.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0d4acb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d015b0d0 con 0x7fe0d00d0010 2024-08-17T02:25:07.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0ceffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d0159040 0x7fe0d015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0ceffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d0159040 0x7fe0d015a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53464/0 (socket says 172.21.15.100:53464) 2024-08-17T02:25:07.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0cf7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d00d0010 0x7fe0d01693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0ceffd700 1 -- 172.21.15.100:0/3357766530 learned_addr learned my addr 172.21.15.100:0/3357766530 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:07.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.608+0000 7fe0cffff700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 0x7fe0d0152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0ceffd700 1 -- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d00d0010 msgr2=0x7fe0d01693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0ceffd700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d00d0010 0x7fe0d01693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0ceffd700 1 -- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 msgr2=0x7fe0d0152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0ceffd700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 0x7fe0d0152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0ceffd700 1 -- 172.21.15.100:0/3357766530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4009420 con 0x7fe0d0159040 2024-08-17T02:25:07.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0ceffd700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d0159040 0x7fe0d015a7d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000c5e0 tx=0x7fe0c0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:07.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0b77fe700 1 -- 172.21.15.100:0/3357766530 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0005b30 con 0x7fe0d0159040 2024-08-17T02:25:07.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0b77fe700 1 -- 172.21.15.100:0/3357766530 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c0005c90 con 0x7fe0d0159040 2024-08-17T02:25:07.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c8009a20 con 0x7fe0d0159040 2024-08-17T02:25:07.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0b77fe700 1 -- 172.21.15.100:0/3357766530 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0007300 con 0x7fe0d0159040 2024-08-17T02:25:07.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d015b690 con 0x7fe0d0159040 2024-08-17T02:25:07.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.612+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d00b0360 con 0x7fe0d0159040 2024-08-17T02:25:07.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.616+0000 7fe0b77fe700 1 -- 172.21.15.100:0/3357766530 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe0c001b3b0 con 0x7fe0d0159040 2024-08-17T02:25:07.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.616+0000 7fe0b77fe700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe0bc060860 0x7fe0bc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.616+0000 7fe0b77fe700 1 -- 172.21.15.100:0/3357766530 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fe0c007c350 con 0x7fe0d0159040 2024-08-17T02:25:07.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.616+0000 7fe0cf7fe700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe0bc060860 0x7fe0bc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.616+0000 7fe0cf7fe700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe0bc060860 0x7fe0bc062d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009f40 tx=0x7fe0c40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:07.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.620+0000 7fe0b77fe700 1 -- 172.21.15.100:0/3357766530 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe0c004c420 con 0x7fe0d0159040 2024-08-17T02:25:07.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.820+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0d00643a0 con 0x7fe0d0159040 2024-08-17T02:25:07.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.820+0000 7fe0b77fe700 1 -- 172.21.15.100:0/3357766530 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5696 (secure 0 0 0) 0x7fe0c004fa40 con 0x7fe0d0159040 2024-08-17T02:25:07.824 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:07.824 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:52.296101+0000","modified":"2024-08-17T02:22:54.942750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4980":{"gid":4980,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3037167008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3037167008},{"type":"v1","addr":"172.21.15.100:6835","nonce":3037167008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:23:05.071644+0000","modified":"2024-08-17T02:23:06.938110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000351990","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:25:07.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe0bc060860 msgr2=0x7fe0bc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe0bc060860 0x7fe0bc062d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009f40 tx=0x7fe0c40091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d0159040 msgr2=0x7fe0d015a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d0159040 0x7fe0d015a7d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000c5e0 tx=0x7fe0c0001070 comp rx=0 tx=0).stop 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 shutdown_connections 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe0bc060860 0x7fe0bc062d10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d015d5b0 0x7fe0d0152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d0159040 0x7fe0d015a7d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 --2- 172.21.15.100:0/3357766530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0d00d0010 0x7fe0d01693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 >> 172.21.15.100:0/3357766530 conn(0x7fe0d0068250 msgr2=0x7fe0d0159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 shutdown_connections 2024-08-17T02:25:07.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.824+0000 7fe0d4acb700 1 -- 172.21.15.100:0/3357766530 wait complete. 2024-08-17T02:25:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 16 2024-08-17T02:25:07.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:25:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b50e53700 1 Processor -- start 2024-08-17T02:25:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b50e53700 1 -- start start 2024-08-17T02:25:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b50e53700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b4c155e50 0x7f4b4c0a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b50e53700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 0x7f4b4c0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b50e53700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b4c06ebe0 0x7f4b4c06f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b50e53700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c06f5a0 con 0x7f4b4c155e50 2024-08-17T02:25:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b50e53700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c0afc10 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b50e53700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c0aed20 con 0x7f4b4c156220 2024-08-17T02:25:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b4c155e50 0x7f4b4c0a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b43fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 0x7f4b4c0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4ad9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b4c06ebe0 0x7f4b4c06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b4c155e50 0x7f4b4c0a9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53480/0 (socket says 172.21.15.100:53480) 2024-08-17T02:25:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 -- 172.21.15.100:0/432921589 learned_addr learned my addr 172.21.15.100:0/432921589 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:07.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 -- 172.21.15.100:0/432921589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 msgr2=0x7f4b4c0aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 --2- 172.21.15.100:0/432921589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 0x7f4b4c0aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 -- 172.21.15.100:0/432921589 >> v2:172.21.15.123:3300/0 conn(0x7f4b4c06ebe0 msgr2=0x7f4b4c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 --2- 172.21.15.100:0/432921589 >> v2:172.21.15.123:3300/0 conn(0x7f4b4c06ebe0 0x7f4b4c06f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 -- 172.21.15.100:0/432921589 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c0af000 con 0x7f4b4c155e50 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.948+0000 7f4b4a59c700 1 --2- 172.21.15.100:0/432921589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c155e50 0x7f4b4c0a9ee0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0ab9c0 tx=0x7f4b3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7099a6552f5ec7 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b49d9b700 1 -- 172.21.15.100:0/432921589 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00b640 con 0x7f4b4c155e50 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b49d9b700 1 -- 172.21.15.100:0/432921589 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c00b7a0 con 0x7f4b4c155e50 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b49d9b700 1 -- 172.21.15.100:0/432921589 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00bb60 con 0x7f4b4c155e50 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- 172.21.15.100:0/432921589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c155e50 msgr2=0x7f4b4c0a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 --2- 172.21.15.100:0/432921589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c155e50 0x7f4b4c0a9ee0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0ab9c0 tx=0x7f4b3c009250 comp rx=0 tx=0).stop 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- 172.21.15.100:0/432921589 shutdown_connections 2024-08-17T02:25:07.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 --2- 172.21.15.100:0/432921589 >> v2:172.21.15.123:3300/0 conn(0x7f4b4c06ebe0 0x7f4b4c06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 --2- 172.21.15.100:0/432921589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 0x7f4b4c0aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 --2- 172.21.15.100:0/432921589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c155e50 0x7f4b4c0a9ee0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- 172.21.15.100:0/432921589 >> 172.21.15.100:0/432921589 conn(0x7f4b4c066e00 msgr2=0x7f4b4c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- 172.21.15.100:0/432921589 shutdown_connections 2024-08-17T02:25:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- 172.21.15.100:0/432921589 wait complete. 2024-08-17T02:25:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 Processor -- start 2024-08-17T02:25:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- start start 2024-08-17T02:25:07.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c06ebe0 0x7f4b4c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 0x7f4b4c0a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c0acd70 0x7f4b4c0a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c0abd60 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c0ac8e0 con 0x7f4b4c0acd70 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.952+0000 7f4b50e53700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c0abfb0 con 0x7f4b4c156220 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c06ebe0 0x7f4b4c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c06ebe0 0x7f4b4c14e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53492/0 (socket says 172.21.15.100:53492) 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 -- 172.21.15.100:0/4264742394 learned_addr learned my addr 172.21.15.100:0/4264742394 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b43fff700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 0x7f4b4c0a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4ad9d700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c0acd70 0x7f4b4c0a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 -- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 msgr2=0x7f4b4c0a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 0x7f4b4c0a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 -- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c0acd70 msgr2=0x7f4b4c0a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c0acd70 0x7f4b4c0a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 -- 172.21.15.100:0/4264742394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c0094f0 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4ad9d700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c0acd70 0x7f4b4c0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b4a59c700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c06ebe0 0x7f4b4c14e910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c006fd0 tx=0x7f4b3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b427fc700 1 -- 172.21.15.100:0/4264742394 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00b910 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b427fc700 1 -- 172.21.15.100:0/4264742394 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c00ba70 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c0ac290 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b427fc700 1 -- 172.21.15.100:0/4264742394 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c011650 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c14ee50 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4c0645c0 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.956+0000 7f4b427fc700 1 -- 172.21.15.100:0/4264742394 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4b3c0119c0 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.960+0000 7f4b427fc700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b30060860 0x7f4b30062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:07.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.960+0000 7f4b427fc700 1 -- 172.21.15.100:0/4264742394 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f4b3c07e610 con 0x7f4b4c06ebe0 2024-08-17T02:25:07.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.960+0000 7f4b43fff700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b30060860 0x7f4b30062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:07.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.960+0000 7f4b43fff700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b30060860 0x7f4b30062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0682a0 tx=0x7f4b34011040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:07.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:07.964+0000 7f4b427fc700 1 -- 172.21.15.100:0/4264742394 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4b3c04e6e0 con 0x7f4b4c06ebe0 2024-08-17T02:25:08.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.124+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b4c099b90 con 0x7f4b4c06ebe0 2024-08-17T02:25:08.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.128+0000 7f4b427fc700 1 -- 172.21.15.100:0/4264742394 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18647 (secure 0 0 0) 0x7f4b3c051d00 con 0x7f4b4c06ebe0 2024-08-17T02:25:08.131 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:08.131 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":38,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:23:08.062746+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:50.544046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:51.275880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000351990.meta","create_time":"2024-08-17T02:23:03.999505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000351990"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000351990.data","create_time":"2024-08-17T02:23:04.098167+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000351990"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:25:08.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b30060860 msgr2=0x7f4b30062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b30060860 0x7f4b30062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0682a0 tx=0x7f4b34011040 comp rx=0 tx=0).stop 2024-08-17T02:25:08.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c06ebe0 msgr2=0x7f4b4c14e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c06ebe0 0x7f4b4c14e910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c006fd0 tx=0x7f4b3c017000 comp rx=0 tx=0).stop 2024-08-17T02:25:08.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 shutdown_connections 2024-08-17T02:25:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f4b30060860 0x7f4b30062d10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b4c0acd70 0x7f4b4c0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b4c156220 0x7f4b4c0a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 --2- 172.21.15.100:0/4264742394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4c06ebe0 0x7f4b4c14e910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 >> 172.21.15.100:0/4264742394 conn(0x7f4b4c066e00 msgr2=0x7f4b4c06a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 shutdown_connections 2024-08-17T02:25:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.132+0000 7f4b50e53700 1 -- 172.21.15.100:0/4264742394 wait complete. 2024-08-17T02:25:08.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:22:50.544046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:22:51.275880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000351990.meta', 'create_time': '2024-08-17T02:23:03.999505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000351990'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000351990.data', 'create_time': '2024-08-17T02:23:04.098167+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000351990'}}}] 2024-08-17T02:25:08.146 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:25:08.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.256+0000 7fc797385700 1 Processor -- start 2024-08-17T02:25:08.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.256+0000 7fc797385700 1 -- start start 2024-08-17T02:25:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7901554c0 0x7fc7900699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79006b000 con 0x7fc790068db0 2024-08-17T02:25:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79006a3f0 con 0x7fc790069ee0 2024-08-17T02:25:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79006a6d0 con 0x7fc7901554c0 2024-08-17T02:25:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc795121700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7901554c0 0x7fc7900699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc795922700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc790068db0 0x7fc79006b490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47368/0 (socket says 172.21.15.100:47368) 2024-08-17T02:25:08.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc795922700 1 -- 172.21.15.100:0/1607097257 learned_addr learned my addr 172.21.15.100:0/1607097257 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc795121700 1 -- 172.21.15.100:0/1607097257 >> v1:172.21.15.123:6789/0 conn(0x7fc790069ee0 legacy=0x7fc7900686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc795121700 1 -- 172.21.15.100:0/1607097257 >> v1:172.21.15.100:6789/0 conn(0x7fc790068db0 legacy=0x7fc79006b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc795121700 1 -- 172.21.15.100:0/1607097257 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79006a9b0 con 0x7fc7901554c0 2024-08-17T02:25:08.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc795121700 1 --2- 172.21.15.100:0/1607097257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900699a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098e0 tx=0x7fc78c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5927acb292d8a525 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc787fff700 1 -- 172.21.15.100:0/1607097257 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c009e80 con 0x7fc7901554c0 2024-08-17T02:25:08.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc787fff700 1 -- 172.21.15.100:0/1607097257 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c00b820 con 0x7fc7901554c0 2024-08-17T02:25:08.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc787fff700 1 -- 172.21.15.100:0/1607097257 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00bba0 con 0x7fc7901554c0 2024-08-17T02:25:08.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 -- 172.21.15.100:0/1607097257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 msgr2=0x7fc7900699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 --2- 172.21.15.100:0/1607097257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900699a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098e0 tx=0x7fc78c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 -- 172.21.15.100:0/1607097257 shutdown_connections 2024-08-17T02:25:08.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 --2- 172.21.15.100:0/1607097257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900699a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 -- 172.21.15.100:0/1607097257 >> 172.21.15.100:0/1607097257 conn(0x7fc7900c9bf0 msgr2=0x7fc79006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:08.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 -- 172.21.15.100:0/1607097257 shutdown_connections 2024-08-17T02:25:08.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.260+0000 7fc797385700 1 -- 172.21.15.100:0/1607097257 wait complete. 2024-08-17T02:25:08.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.264+0000 7fc797385700 1 Processor -- start 2024-08-17T02:25:08.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.264+0000 7fc797385700 1 -- start start 2024-08-17T02:25:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc790068db0 0x7fc7901685d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc790069ee0 0x7fc7900aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc790146a90 con 0x7fc7901554c0 2024-08-17T02:25:08.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900a6d20 con 0x7fc790068db0 2024-08-17T02:25:08.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900a6110 con 0x7fc790069ee0 2024-08-17T02:25:08.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53514/0 (socket says 172.21.15.100:53514) 2024-08-17T02:25:08.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 -- 172.21.15.100:0/2990694571 learned_addr learned my addr 172.21.15.100:0/2990694571 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:08.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795121700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc790068db0 0x7fc7901685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc794920700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc790069ee0 0x7fc7900aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 -- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc790069ee0 msgr2=0x7fc7900aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc790069ee0 0x7fc7900aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 -- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc790068db0 msgr2=0x7fc7901685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc790068db0 0x7fc7901685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 -- 172.21.15.100:0/2990694571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c009420 con 0x7fc7901554c0 2024-08-17T02:25:08.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc795922700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900aae10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc78800cc70 tx=0x7fc7880019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc7857fa700 1 -- 172.21.15.100:0/2990694571 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78800a830 con 0x7fc7901554c0 2024-08-17T02:25:08.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc7857fa700 1 -- 172.21.15.100:0/2990694571 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78800a9b0 con 0x7fc7901554c0 2024-08-17T02:25:08.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7900a63f0 con 0x7fc7901554c0 2024-08-17T02:25:08.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc7857fa700 1 -- 172.21.15.100:0/2990694571 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78800ad70 con 0x7fc7901554c0 2024-08-17T02:25:08.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7900a69b0 con 0x7fc7901554c0 2024-08-17T02:25:08.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.268+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7900a6eb0 con 0x7fc7901554c0 2024-08-17T02:25:08.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.272+0000 7fc7857fa700 1 -- 172.21.15.100:0/2990694571 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc788007300 con 0x7fc7901554c0 2024-08-17T02:25:08.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.272+0000 7fc7857fa700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc774060860 0x7fc774062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.272+0000 7fc7857fa700 1 -- 172.21.15.100:0/2990694571 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fc78807d1d0 con 0x7fc7901554c0 2024-08-17T02:25:08.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.272+0000 7fc795121700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc774060860 0x7fc774062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.272+0000 7fc795121700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc774060860 0x7fc774062d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0030c0 tx=0x7fc78c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.276+0000 7fc7857fa700 1 -- 172.21.15.100:0/2990694571 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc78804d320 con 0x7fc7901554c0 2024-08-17T02:25:08.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.436+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7900645c0 con 0x7fc7901554c0 2024-08-17T02:25:08.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.436+0000 7fc7857fa700 1 -- 172.21.15.100:0/2990694571 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc788050940 con 0x7fc7901554c0 2024-08-17T02:25:08.440 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:25:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc774060860 msgr2=0x7fc774062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc774060860 0x7fc774062d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0030c0 tx=0x7fc78c016000 comp rx=0 tx=0).stop 2024-08-17T02:25:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 msgr2=0x7fc7900aae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900aae10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc78800cc70 tx=0x7fc7880019f0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 shutdown_connections 2024-08-17T02:25:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc774060860 0x7fc774062d10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7901554c0 0x7fc7900aae10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc790069ee0 0x7fc7900aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 --2- 172.21.15.100:0/2990694571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc790068db0 0x7fc7901685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 >> 172.21.15.100:0/2990694571 conn(0x7fc7900c9bf0 msgr2=0x7fc7900af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 shutdown_connections 2024-08-17T02:25:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.440+0000 7fc797385700 1 -- 172.21.15.100:0/2990694571 wait complete. 2024-08-17T02:25:08.453 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:25:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb2a91700 1 Processor -- start 2024-08-17T02:25:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb2a91700 1 -- start start 2024-08-17T02:25:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb2a91700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcac067030 0x7fdcac067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb2a91700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcac0cf670 con 0x7fdcac158cc0 2024-08-17T02:25:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb2a91700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcac0cea60 con 0x7fdcac0cfb00 2024-08-17T02:25:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb2a91700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcac0ced40 con 0x7fdcac067030 2024-08-17T02:25:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb2290700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdcac158cc0 0x7fdcac1590d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47376/0 (socket says 172.21.15.100:47376) 2024-08-17T02:25:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb2290700 1 -- 172.21.15.100:0/3363411766 learned_addr learned my addr 172.21.15.100:0/3363411766 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.564+0000 7fdcb1a8f700 1 --2- 172.21.15.100:0/3363411766 >> v2:172.21.15.123:3300/0 conn(0x7fdcac067030 0x7fdcac067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb0a8d700 1 -- 172.21.15.100:0/3363411766 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270552341 0 0) 0x7fdcac0cf670 con 0x7fdcac158cc0 2024-08-17T02:25:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb0a8d700 1 -- 172.21.15.100:0/3363411766 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc94003570 con 0x7fdcac158cc0 2024-08-17T02:25:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb1a8f700 1 -- 172.21.15.100:0/3363411766 >> v1:172.21.15.123:6789/0 conn(0x7fdcac0cfb00 legacy=0x7fdcac0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb1a8f700 1 -- 172.21.15.100:0/3363411766 >> v1:172.21.15.100:6789/0 conn(0x7fdcac158cc0 legacy=0x7fdcac1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb1a8f700 1 -- 172.21.15.100:0/3363411766 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac0cf020 con 0x7fdcac067030 2024-08-17T02:25:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb1a8f700 1 --2- 172.21.15.100:0/3363411766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac067030 0x7fdcac067400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009e60 tx=0x7fdc9c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851b6a4c815433b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb0a8d700 1 -- 172.21.15.100:0/3363411766 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00e070 con 0x7fdcac067030 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb0a8d700 1 -- 172.21.15.100:0/3363411766 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c00bb60 con 0x7fdcac067030 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb0a8d700 1 -- 172.21.15.100:0/3363411766 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c012430 con 0x7fdcac067030 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3363411766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac067030 msgr2=0x7fdcac067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb2a91700 1 --2- 172.21.15.100:0/3363411766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac067030 0x7fdcac067400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009e60 tx=0x7fdc9c009780 comp rx=0 tx=0).stop 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3363411766 shutdown_connections 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb2a91700 1 --2- 172.21.15.100:0/3363411766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac067030 0x7fdcac067400 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3363411766 >> 172.21.15.100:0/3363411766 conn(0x7fdcac0d0b50 msgr2=0x7fdcac0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3363411766 shutdown_connections 2024-08-17T02:25:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3363411766 wait complete. 2024-08-17T02:25:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.568+0000 7fdcb2a91700 1 Processor -- start 2024-08-17T02:25:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 -- start start 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcac067030 0x7fdcac155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac0cfb00 0x7fdcac158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac158cc0 0x7fdcac152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac155b70 con 0x7fdcac158cc0 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac157020 con 0x7fdcac0cfb00 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1572c0 con 0x7fdcac067030 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac158cc0 0x7fdcac152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac158cc0 0x7fdcac152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53518/0 (socket says 172.21.15.100:53518) 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 -- 172.21.15.100:0/3316834018 learned_addr learned my addr 172.21.15.100:0/3316834018 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb1a8f700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcac067030 0x7fdcac155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 -- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcac067030 msgr2=0x7fdcac155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcac067030 0x7fdcac155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 -- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac0cfb00 msgr2=0x7fdcac158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb128e700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac0cfb00 0x7fdcac158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac0cfb00 0x7fdcac158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 -- 172.21.15.100:0/3316834018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c009a20 con 0x7fdcac158cc0 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb1a8f700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcac067030 0x7fdcac155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb128e700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac0cfb00 0x7fdcac158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2290700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac158cc0 0x7fdcac152fb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdca40010e0 tx=0x7fdca4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdca9ffb700 1 -- 172.21.15.100:0/3316834018 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4005c00 con 0x7fdcac158cc0 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdca9ffb700 1 -- 172.21.15.100:0/3316834018 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca4005d60 con 0x7fdcac158cc0 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac157880 con 0x7fdcac158cc0 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdca9ffb700 1 -- 172.21.15.100:0/3316834018 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca40072e0 con 0x7fdcac158cc0 2024-08-17T02:25:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.572+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac155880 con 0x7fdcac158cc0 2024-08-17T02:25:08.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.576+0000 7fdca9ffb700 1 -- 172.21.15.100:0/3316834018 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdca4012400 con 0x7fdcac158cc0 2024-08-17T02:25:08.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.576+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcac0638f0 con 0x7fdcac158cc0 2024-08-17T02:25:08.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.576+0000 7fdca9ffb700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdc98060860 0x7fdc98062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.576+0000 7fdca9ffb700 1 -- 172.21.15.100:0/3316834018 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fdca407c3d0 con 0x7fdcac158cc0 2024-08-17T02:25:08.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.576+0000 7fdcb1a8f700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdc98060860 0x7fdc98062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.576+0000 7fdcb1a8f700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdc98060860 0x7fdc98062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009e60 tx=0x7fdc9c019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.580+0000 7fdca9ffb700 1 -- 172.21.15.100:0/3316834018 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdca404a480 con 0x7fdcac158cc0 2024-08-17T02:25:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.744+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcac154600 con 0x7fdcac158cc0 2024-08-17T02:25:08.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.744+0000 7fdca9ffb700 1 -- 172.21.15.100:0/3316834018 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdca404c4a0 con 0x7fdcac158cc0 2024-08-17T02:25:08.747 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:25:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdc98060860 msgr2=0x7fdc98062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdc98060860 0x7fdc98062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009e60 tx=0x7fdc9c019000 comp rx=0 tx=0).stop 2024-08-17T02:25:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac158cc0 msgr2=0x7fdcac152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac158cc0 0x7fdcac152fb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdca40010e0 tx=0x7fdca4007dd0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 shutdown_connections 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdc98060860 0x7fdc98062d10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac158cc0 0x7fdcac152fb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcac0cfb00 0x7fdcac158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 --2- 172.21.15.100:0/3316834018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcac067030 0x7fdcac155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 >> 172.21.15.100:0/3316834018 conn(0x7fdcac0d0b50 msgr2=0x7fdcac0b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 shutdown_connections 2024-08-17T02:25:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.748+0000 7fdcb2a91700 1 -- 172.21.15.100:0/3316834018 wait complete. 2024-08-17T02:25:08.762 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:25:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb0b2a9700 1 Processor -- start 2024-08-17T02:25:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb0b2a9700 1 -- start start 2024-08-17T02:25:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb0b2a9700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb040cfb00 0x7ffb04067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb0b2a9700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb040cf670 con 0x7ffb04067030 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb0b2a9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb040cea60 con 0x7ffb04158cc0 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb0b2a9700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb040ced40 con 0x7ffb040cfb00 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb0a2a7700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ffb04067030 0x7ffb040cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47378/0 (socket says 172.21.15.100:47378) 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb09aa6700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb040cfb00 0x7ffb04067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb0a2a7700 1 -- 172.21.15.100:0/741187968 learned_addr learned my addr 172.21.15.100:0/741187968 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb092a5700 1 -- 172.21.15.100:0/741187968 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997147851 0 0) 0x7ffb040cf670 con 0x7ffb04067030 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb092a5700 1 -- 172.21.15.100:0/741187968 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf0003570 con 0x7ffb04067030 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb09aa6700 1 -- 172.21.15.100:0/741187968 >> v1:172.21.15.123:6789/0 conn(0x7ffb04158cc0 legacy=0x7ffb041590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb09aa6700 1 -- 172.21.15.100:0/741187968 >> v1:172.21.15.100:6789/0 conn(0x7ffb04067030 legacy=0x7ffb040cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb09aa6700 1 -- 172.21.15.100:0/741187968 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040cf020 con 0x7ffb040cfb00 2024-08-17T02:25:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb09aa6700 1 --2- 172.21.15.100:0/741187968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb040cfb00 0x7ffb04067400 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40098e0 tx=0x7ffaf40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76ff313efc2dcd8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb092a5700 1 -- 172.21.15.100:0/741187968 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400b640 con 0x7ffb040cfb00 2024-08-17T02:25:08.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb092a5700 1 -- 172.21.15.100:0/741187968 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf400b7a0 con 0x7ffb040cfb00 2024-08-17T02:25:08.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.868+0000 7ffb092a5700 1 -- 172.21.15.100:0/741187968 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400bb40 con 0x7ffb040cfb00 2024-08-17T02:25:08.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/741187968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb040cfb00 msgr2=0x7ffb04067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 --2- 172.21.15.100:0/741187968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb040cfb00 0x7ffb04067400 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40098e0 tx=0x7ffaf40091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/741187968 shutdown_connections 2024-08-17T02:25:08.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 --2- 172.21.15.100:0/741187968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb040cfb00 0x7ffb04067400 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/741187968 >> 172.21.15.100:0/741187968 conn(0x7ffb040d0b50 msgr2=0x7ffb040ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:08.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/741187968 shutdown_connections 2024-08-17T02:25:08.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/741187968 wait complete. 2024-08-17T02:25:08.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 Processor -- start 2024-08-17T02:25:08.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- start start 2024-08-17T02:25:08.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04067030 0x7ffb04142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb040cfb00 0x7ffb040b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb04158cc0 0x7ffb040b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0415d750 con 0x7ffb04158cc0 2024-08-17T02:25:08.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0415cb40 con 0x7ffb04067030 2024-08-17T02:25:08.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0b2a9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0415ce20 con 0x7ffb040cfb00 2024-08-17T02:25:08.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0aaa8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb04158cc0 0x7ffb040b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0aaa8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb04158cc0 0x7ffb040b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53542/0 (socket says 172.21.15.100:53542) 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0aaa8700 1 -- 172.21.15.100:0/1743045394 learned_addr learned my addr 172.21.15.100:0/1743045394 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0a2a7700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04067030 0x7ffb04142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb09aa6700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb040cfb00 0x7ffb040b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0aaa8700 1 -- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb040cfb00 msgr2=0x7ffb040b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0aaa8700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb040cfb00 0x7ffb040b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0aaa8700 1 -- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04067030 msgr2=0x7ffb04142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0aaa8700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04067030 0x7ffb04142e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.872+0000 7ffb0aaa8700 1 -- 172.21.15.100:0/1743045394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4009420 con 0x7ffb04158cc0 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffb0aaa8700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb04158cc0 0x7ffb040b45b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00c840 tx=0x7ffafc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffafa7fc700 1 -- 172.21.15.100:0/1743045394 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc012630 con 0x7ffb04158cc0 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffafa7fc700 1 -- 172.21.15.100:0/1743045394 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc0127b0 con 0x7ffb04158cc0 2024-08-17T02:25:08.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0415d100 con 0x7ffb04158cc0 2024-08-17T02:25:08.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffafa7fc700 1 -- 172.21.15.100:0/1743045394 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc012b70 con 0x7ffb04158cc0 2024-08-17T02:25:08.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb04143360 con 0x7ffb04158cc0 2024-08-17T02:25:08.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0415d8c0 con 0x7ffb04158cc0 2024-08-17T02:25:08.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffafa7fc700 1 -- 172.21.15.100:0/1743045394 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ffafc01a3a0 con 0x7ffb04158cc0 2024-08-17T02:25:08.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffafa7fc700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffae8060860 0x7ffae8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:08.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffafa7fc700 1 -- 172.21.15.100:0/1743045394 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7ffafc07d030 con 0x7ffb04158cc0 2024-08-17T02:25:08.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.876+0000 7ffb0a2a7700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffae8060860 0x7ffae8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:08.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.880+0000 7ffb0a2a7700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffae8060860 0x7ffae8062d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffb00007fd0 tx=0x7ffb00009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:08.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:08.884+0000 7ffafa7fc700 1 -- 172.21.15.100:0/1743045394 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ffafc04d100 con 0x7ffb04158cc0 2024-08-17T02:25:09.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:09.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:09.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb040638f0 con 0x7ffb04158cc0 2024-08-17T02:25:09.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffafa7fc700 1 -- 172.21.15.100:0/1743045394 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffafc050720 con 0x7ffb04158cc0 2024-08-17T02:25:09.050 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-08-17T02:25:09.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffae8060860 msgr2=0x7ffae8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffae8060860 0x7ffae8062d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffb00007fd0 tx=0x7ffb00009040 comp rx=0 tx=0).stop 2024-08-17T02:25:09.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb04158cc0 msgr2=0x7ffb040b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb04158cc0 0x7ffb040b45b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00c840 tx=0x7ffafc001670 comp rx=0 tx=0).stop 2024-08-17T02:25:09.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 shutdown_connections 2024-08-17T02:25:09.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffae8060860 0x7ffae8062d10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb04158cc0 0x7ffb040b45b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb040cfb00 0x7ffb040b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 --2- 172.21.15.100:0/1743045394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04067030 0x7ffb04142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.048+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 >> 172.21.15.100:0/1743045394 conn(0x7ffb040d0b50 msgr2=0x7ffb040c5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:09.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.052+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 shutdown_connections 2024-08-17T02:25:09.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.052+0000 7ffb0b2a9700 1 -- 172.21.15.100:0/1743045394 wait complete. 2024-08-17T02:25:09.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000351990.meta pg_num 2024-08-17T02:25:09.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.228+0000 7f27d913b700 1 Processor -- start 2024-08-17T02:25:09.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.228+0000 7f27d913b700 1 -- start start 2024-08-17T02:25:09.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27d41590a0 0x7f27d4159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cf9b0 0x7f27d40cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d40cf520 con 0x7f27d40cb600 2024-08-17T02:25:09.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d40d0f40 con 0x7f27d41590a0 2024-08-17T02:25:09.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d40ce690 con 0x7f27d40cf9b0 2024-08-17T02:25:09.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d3fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27d41590a0 0x7f27d4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d3fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27d41590a0 0x7f27d4159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53550/0 (socket says 172.21.15.100:53550) 2024-08-17T02:25:09.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d3fff700 1 -- 172.21.15.100:0/1327094358 learned_addr learned my addr 172.21.15.100:0/1327094358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:09.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d37fe700 1 --2- 172.21.15.100:0/1327094358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cf9b0 0x7f27d40cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d3fff700 1 -- 172.21.15.100:0/1327094358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cf9b0 msgr2=0x7f27d40cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d3fff700 1 --2- 172.21.15.100:0/1327094358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cf9b0 0x7f27d40cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d3fff700 1 -- 172.21.15.100:0/1327094358 >> v1:172.21.15.123:6789/0 conn(0x7f27d40cb600 legacy=0x7f27d4066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d3fff700 1 -- 172.21.15.100:0/1327094358 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d40ce910 con 0x7f27d41590a0 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d3fff700 1 --2- 172.21.15.100:0/1327094358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d41590a0 0x7f27d4159470 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f27c80099b0 tx=0x7f27c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1276e1e4abf58e24 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d2ffd700 1 -- 172.21.15.100:0/1327094358 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8009ea0 con 0x7f27d41590a0 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d2ffd700 1 -- 172.21.15.100:0/1327094358 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c800b860 con 0x7f27d41590a0 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d2ffd700 1 -- 172.21.15.100:0/1327094358 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c800bc00 con 0x7f27d41590a0 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 -- 172.21.15.100:0/1327094358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d41590a0 msgr2=0x7f27d4159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 --2- 172.21.15.100:0/1327094358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d41590a0 0x7f27d4159470 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f27c80099b0 tx=0x7f27c8009250 comp rx=0 tx=0).stop 2024-08-17T02:25:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 -- 172.21.15.100:0/1327094358 shutdown_connections 2024-08-17T02:25:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 --2- 172.21.15.100:0/1327094358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cf9b0 0x7f27d40cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 --2- 172.21.15.100:0/1327094358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d41590a0 0x7f27d4159470 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 -- 172.21.15.100:0/1327094358 >> 172.21.15.100:0/1327094358 conn(0x7f27d4068c80 msgr2=0x7f27d40d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 -- 172.21.15.100:0/1327094358 shutdown_connections 2024-08-17T02:25:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.232+0000 7f27d913b700 1 -- 172.21.15.100:0/1327094358 wait complete. 2024-08-17T02:25:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.236+0000 7f27d913b700 1 Processor -- start 2024-08-17T02:25:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.236+0000 7f27d913b700 1 -- start start 2024-08-17T02:25:09.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cb600 0x7f27d4152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40cf9b0 0x7f27d415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d415a910 0x7f27d415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d40b3300 con 0x7f27d415a910 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d415bad0 con 0x7f27d40cf9b0 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d415aec0 con 0x7f27d40cb600 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d415a910 0x7f27d415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40cf9b0 0x7f27d415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cb600 0x7f27d4152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d415a910 0x7f27d415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53566/0 (socket says 172.21.15.100:53566) 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 -- 172.21.15.100:0/390357051 learned_addr learned my addr 172.21.15.100:0/390357051 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 -- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cb600 msgr2=0x7f27d4152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cb600 0x7f27d4152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 -- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40cf9b0 msgr2=0x7f27d415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40cf9b0 0x7f27d415a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 -- 172.21.15.100:0/390357051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80094f0 con 0x7f27d415a910 2024-08-17T02:25:09.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d893a700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d415a910 0x7f27d415ed90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00c910 tx=0x7f27cc0016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27bbfff700 1 -- 172.21.15.100:0/390357051 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc012630 con 0x7f27d415a910 2024-08-17T02:25:09.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27bbfff700 1 -- 172.21.15.100:0/390357051 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc0127b0 con 0x7f27d415a910 2024-08-17T02:25:09.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d415b1a0 con 0x7f27d415a910 2024-08-17T02:25:09.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27bbfff700 1 -- 172.21.15.100:0/390357051 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc012b70 con 0x7f27d415a910 2024-08-17T02:25:09.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d415b760 con 0x7f27d415a910 2024-08-17T02:25:09.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d4064330 con 0x7f27d415a910 2024-08-17T02:25:09.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.240+0000 7f27bbfff700 1 -- 172.21.15.100:0/390357051 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f27cc024420 con 0x7f27d415a910 2024-08-17T02:25:09.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.244+0000 7f27bbfff700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27c0060860 0x7f27c0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.244+0000 7f27bbfff700 1 -- 172.21.15.100:0/390357051 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f27cc07d7a0 con 0x7f27d415a910 2024-08-17T02:25:09.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.244+0000 7f27d3fff700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27c0060860 0x7f27c0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.244+0000 7f27d3fff700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27c0060860 0x7f27c0062d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f27c80099b0 tx=0x7f27c8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.248+0000 7f27bbfff700 1 -- 172.21.15.100:0/390357051 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f27cc04d870 con 0x7f27d415a910 2024-08-17T02:25:09.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000351990.meta", "var": "pg_num"} v 0) v1 -- 0x7f27d4067b80 con 0x7f27d415a910 2024-08-17T02:25:09.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27bbfff700 1 -- 172.21.15.100:0/390357051 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000351990.meta", "var": "pg_num"}]=0 v38) v1 ==== 124+0+11 (secure 0 0 0) 0x7f27cc050e90 con 0x7f27d415a910 2024-08-17T02:25:09.418 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:25:09.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27c0060860 msgr2=0x7f27c0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27c0060860 0x7f27c0062d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f27c80099b0 tx=0x7f27c8017000 comp rx=0 tx=0).stop 2024-08-17T02:25:09.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d415a910 msgr2=0x7f27d415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d415a910 0x7f27d415ed90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00c910 tx=0x7f27cc0016b0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 shutdown_connections 2024-08-17T02:25:09.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27c0060860 0x7f27c0062d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d415a910 0x7f27d415ed90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40cf9b0 0x7f27d415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 --2- 172.21.15.100:0/390357051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40cb600 0x7f27d4152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 >> 172.21.15.100:0/390357051 conn(0x7f27d4068c80 msgr2=0x7f27d4154740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:09.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 shutdown_connections 2024-08-17T02:25:09.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.416+0000 7f27d913b700 1 -- 172.21.15.100:0/390357051 wait complete. 2024-08-17T02:25:09.437 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000351990.data pg_num 2024-08-17T02:25:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.548+0000 7fb10f0ab700 1 Processor -- start 2024-08-17T02:25:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.548+0000 7fb10f0ab700 1 -- start start 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.548+0000 7fb10f0ab700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb10806eea0 0x7fb1080b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.548+0000 7fb10f0ab700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1080ace30 con 0x7fb108152bd0 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.548+0000 7fb10f0ab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb108156dd0 con 0x7fb10806ead0 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.548+0000 7fb10f0ab700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb108156450 con 0x7fb10806eea0 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10e8aa700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb108152bd0 0x7fb108152ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47394/0 (socket says 172.21.15.100:47394) 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10d8a8700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb10806eea0 0x7fb1080b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10e8aa700 1 -- 172.21.15.100:0/4125934356 learned_addr learned my addr 172.21.15.100:0/4125934356 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10d8a8700 1 -- 172.21.15.100:0/4125934356 >> v1:172.21.15.123:6789/0 conn(0x7fb10806ead0 legacy=0x7fb108155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10d8a8700 1 -- 172.21.15.100:0/4125934356 >> v1:172.21.15.100:6789/0 conn(0x7fb108152bd0 legacy=0x7fb108152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10d8a8700 1 -- 172.21.15.100:0/4125934356 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108156730 con 0x7fb10806eea0 2024-08-17T02:25:09.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10d8a8700 1 --2- 172.21.15.100:0/4125934356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806eea0 0x7fb1080b1830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009e60 tx=0x7fb0f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5fa805faa57acb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10d0a7700 1 -- 172.21.15.100:0/4125934356 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800e070 con 0x7fb10806eea0 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10d0a7700 1 -- 172.21.15.100:0/4125934356 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f800bc30 con 0x7fb10806eea0 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10d0a7700 1 -- 172.21.15.100:0/4125934356 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8012430 con 0x7fb10806eea0 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10f0ab700 1 -- 172.21.15.100:0/4125934356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806eea0 msgr2=0x7fb1080b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10f0ab700 1 --2- 172.21.15.100:0/4125934356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806eea0 0x7fb1080b1830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009e60 tx=0x7fb0f8009780 comp rx=0 tx=0).stop 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10f0ab700 1 -- 172.21.15.100:0/4125934356 shutdown_connections 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10f0ab700 1 --2- 172.21.15.100:0/4125934356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806eea0 0x7fb1080b1830 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10f0ab700 1 -- 172.21.15.100:0/4125934356 >> 172.21.15.100:0/4125934356 conn(0x7fb1080678a0 msgr2=0x7fb108067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10f0ab700 1 -- 172.21.15.100:0/4125934356 shutdown_connections 2024-08-17T02:25:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.552+0000 7fb10f0ab700 1 -- 172.21.15.100:0/4125934356 wait complete. 2024-08-17T02:25:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10f0ab700 1 Processor -- start 2024-08-17T02:25:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10f0ab700 1 -- start start 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10f0ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806ead0 0x7fb1081485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10f0ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb10806eea0 0x7fb10806a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10f0ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb108152bd0 0x7fb108143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10f0ab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080a5450 con 0x7fb10806ead0 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10f0ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080a56a0 con 0x7fb10806eea0 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10f0ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080a5980 con 0x7fb108152bd0 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806ead0 0x7fb1081485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806ead0 0x7fb1081485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53586/0 (socket says 172.21.15.100:53586) 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 -- 172.21.15.100:0/2955140122 learned_addr learned my addr 172.21.15.100:0/2955140122 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10d8a8700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb10806eea0 0x7fb10806a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e8aa700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb108152bd0 0x7fb108143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 -- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb108152bd0 msgr2=0x7fb108143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb108152bd0 0x7fb108143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 -- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb10806eea0 msgr2=0x7fb10806a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb10806eea0 0x7fb10806a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 -- 172.21.15.100:0/2955140122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f8009a20 con 0x7fb10806ead0 2024-08-17T02:25:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e8aa700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb108152bd0 0x7fb108143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.556+0000 7fb10e0a9700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806ead0 0x7fb1081485d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c590 tx=0x7fb0fc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb1067fc700 1 -- 172.21.15.100:0/2955140122 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc011590 con 0x7fb10806ead0 2024-08-17T02:25:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb1067fc700 1 -- 172.21.15.100:0/2955140122 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc011730 con 0x7fb10806ead0 2024-08-17T02:25:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1080a5c60 con 0x7fb10806ead0 2024-08-17T02:25:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb1067fc700 1 -- 172.21.15.100:0/2955140122 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc011af0 con 0x7fb10806ead0 2024-08-17T02:25:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb108148b10 con 0x7fb10806ead0 2024-08-17T02:25:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1080a9100 con 0x7fb10806ead0 2024-08-17T02:25:09.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb1067fc700 1 -- 172.21.15.100:0/2955140122 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb0fc012420 con 0x7fb10806ead0 2024-08-17T02:25:09.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb1067fc700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb0f4060890 0x7fb0f4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb1067fc700 1 -- 172.21.15.100:0/2955140122 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fb0fc07c260 con 0x7fb10806ead0 2024-08-17T02:25:09.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.560+0000 7fb10d8a8700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb0f4060890 0x7fb0f4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.564+0000 7fb10d8a8700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb0f4060890 0x7fb0f4062d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009e60 tx=0x7fb0f8019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.568+0000 7fb1067fc700 1 -- 172.21.15.100:0/2955140122 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb0fc04c3e0 con 0x7fb10806ead0 2024-08-17T02:25:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000351990.data", "var": "pg_num"} v 0) v1 -- 0x7fb108064330 con 0x7fb10806ead0 2024-08-17T02:25:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb1067fc700 1 -- 172.21.15.100:0/2955140122 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000351990.data", "var": "pg_num"}]=0 v38) v1 ==== 124+0+11 (secure 0 0 0) 0x7fb0fc017020 con 0x7fb10806ead0 2024-08-17T02:25:09.733 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:25:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb0f4060890 msgr2=0x7fb0f4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb0f4060890 0x7fb0f4062d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009e60 tx=0x7fb0f8019000 comp rx=0 tx=0).stop 2024-08-17T02:25:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806ead0 msgr2=0x7fb1081485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806ead0 0x7fb1081485d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c590 tx=0x7fb0fc007f30 comp rx=0 tx=0).stop 2024-08-17T02:25:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 shutdown_connections 2024-08-17T02:25:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb0f4060890 0x7fb0f4062d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb108152bd0 0x7fb108143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb10806eea0 0x7fb10806a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 --2- 172.21.15.100:0/2955140122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10806ead0 0x7fb1081485d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 >> 172.21.15.100:0/2955140122 conn(0x7fb1080678a0 msgr2=0x7fb108154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 shutdown_connections 2024-08-17T02:25:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.732+0000 7fb10f0ab700 1 -- 172.21.15.100:0/2955140122 wait complete. 2024-08-17T02:25:09.748 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:25:09.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8c59db700 1 Processor -- start 2024-08-17T02:25:09.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8c59db700 1 -- start start 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8c59db700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8c0154900 0x7fe8c00b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8c59db700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00b12d0 0x7fe8c00cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8c59db700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c00cbaa0 con 0x7fe8c00cb390 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8c59db700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c0068790 con 0x7fe8c0154900 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8c59db700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c01583f0 con 0x7fe8c00b12d0 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00b12d0 0x7fe8c00cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00b12d0 0x7fe8c00cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:43306/0 (socket says 172.21.15.100:43306) 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bffff700 1 -- 172.21.15.100:0/1109710094 learned_addr learned my addr 172.21.15.100:0/1109710094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8c49d9700 1 --2- 172.21.15.100:0/1109710094 >> v2:172.21.15.123:3300/0 conn(0x7fe8c0154900 0x7fe8c00b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bf7fe700 1 -- 172.21.15.100:0/1109710094 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734318337 0 0) 0x7fe8c00cbaa0 con 0x7fe8c00cb390 2024-08-17T02:25:09.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bffff700 1 -- 172.21.15.100:0/1109710094 >> v2:172.21.15.123:3300/0 conn(0x7fe8c0154900 msgr2=0x7fe8c00b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:09.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bffff700 1 --2- 172.21.15.100:0/1109710094 >> v2:172.21.15.123:3300/0 conn(0x7fe8c0154900 0x7fe8c00b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bffff700 1 -- 172.21.15.100:0/1109710094 >> v1:172.21.15.123:6789/0 conn(0x7fe8c00cb390 legacy=0x7fe8c006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bffff700 1 -- 172.21.15.100:0/1109710094 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00b1700 con 0x7fe8c00b12d0 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.852+0000 7fe8bffff700 1 --2- 172.21.15.100:0/1109710094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00b12d0 0x7fe8c00cae50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009f20 tx=0x7fe8b0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eba1b3b3dbb6d1de server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8bf7fe700 1 -- 172.21.15.100:0/1109710094 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000e070 con 0x7fe8c00b12d0 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8bf7fe700 1 -- 172.21.15.100:0/1109710094 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b000b9d0 con 0x7fe8c00b12d0 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8bf7fe700 1 -- 172.21.15.100:0/1109710094 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00092e0 con 0x7fe8c00b12d0 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 -- 172.21.15.100:0/1109710094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00b12d0 msgr2=0x7fe8c00cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 --2- 172.21.15.100:0/1109710094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00b12d0 0x7fe8c00cae50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009f20 tx=0x7fe8b0009840 comp rx=0 tx=0).stop 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 -- 172.21.15.100:0/1109710094 shutdown_connections 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 --2- 172.21.15.100:0/1109710094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00b12d0 0x7fe8c00cae50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 --2- 172.21.15.100:0/1109710094 >> v2:172.21.15.123:3300/0 conn(0x7fe8c0154900 0x7fe8c00b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 -- 172.21.15.100:0/1109710094 >> 172.21.15.100:0/1109710094 conn(0x7fe8c00678d0 msgr2=0x7fe8c0067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 -- 172.21.15.100:0/1109710094 shutdown_connections 2024-08-17T02:25:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 -- 172.21.15.100:0/1109710094 wait complete. 2024-08-17T02:25:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 Processor -- start 2024-08-17T02:25:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.856+0000 7fe8c59db700 1 -- start start 2024-08-17T02:25:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8c00b12d0 0x7fe8c016b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00cb390 0x7fe8c0152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c0154900 0x7fe8c014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c014f550 con 0x7fe8c0154900 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0150140 con 0x7fe8c00b12d0 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c014f270 con 0x7fe8c00cb390 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c0154900 0x7fe8c014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c0154900 0x7fe8c014cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53602/0 (socket says 172.21.15.100:53602) 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 -- 172.21.15.100:0/92866106 learned_addr learned my addr 172.21.15.100:0/92866106 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c49d9700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8c00b12d0 0x7fe8c016b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8bffff700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00cb390 0x7fe8c0152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 -- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00cb390 msgr2=0x7fe8c0152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00cb390 0x7fe8c0152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 -- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8c00b12d0 msgr2=0x7fe8c016b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8c00b12d0 0x7fe8c016b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 -- 172.21.15.100:0/92866106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b0009ae0 con 0x7fe8c0154900 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c51da700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c0154900 0x7fe8c014cee0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c5d0 tx=0x7fe8b8007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8bcff9700 1 -- 172.21.15.100:0/92866106 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8011590 con 0x7fe8c0154900 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8bcff9700 1 -- 172.21.15.100:0/92866106 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b80116f0 con 0x7fe8c0154900 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c016a860 con 0x7fe8c0154900 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8bcff9700 1 -- 172.21.15.100:0/92866106 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8011ab0 con 0x7fe8c0154900 2024-08-17T02:25:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c016ae20 con 0x7fe8c0154900 2024-08-17T02:25:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.860+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c016b010 con 0x7fe8c0154900 2024-08-17T02:25:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.864+0000 7fe8bcff9700 1 -- 172.21.15.100:0/92866106 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe8b801b3b0 con 0x7fe8c0154900 2024-08-17T02:25:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.864+0000 7fe8bcff9700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe8a0060860 0x7fe8a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.864+0000 7fe8bcff9700 1 -- 172.21.15.100:0/92866106 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fe8b807c380 con 0x7fe8c0154900 2024-08-17T02:25:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.864+0000 7fe8c49d9700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe8a0060860 0x7fe8a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:09.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.864+0000 7fe8c49d9700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe8a0060860 0x7fe8a0062d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0063aa0 tx=0x7fe8a80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:09.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:09.868+0000 7fe8bcff9700 1 -- 172.21.15.100:0/92866106 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe8b804c450 con 0x7fe8c0154900 2024-08-17T02:25:10.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.060+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8c00643a0 con 0x7fe8c0154900 2024-08-17T02:25:10.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.060+0000 7fe8bcff9700 1 -- 172.21.15.100:0/92866106 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5696 (secure 0 0 0) 0x7fe8b804fa70 con 0x7fe8c0154900 2024-08-17T02:25:10.064 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:10.064 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:52.296101+0000","modified":"2024-08-17T02:22:54.942750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4980":{"gid":4980,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3037167008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3037167008},{"type":"v1","addr":"172.21.15.100:6835","nonce":3037167008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:23:05.071644+0000","modified":"2024-08-17T02:23:06.938110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000351990","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:25:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe8a0060860 msgr2=0x7fe8a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe8a0060860 0x7fe8a0062d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0063aa0 tx=0x7fe8a80091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c0154900 msgr2=0x7fe8c014cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c0154900 0x7fe8c014cee0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c5d0 tx=0x7fe8b8007d70 comp rx=0 tx=0).stop 2024-08-17T02:25:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 shutdown_connections 2024-08-17T02:25:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe8a0060860 0x7fe8a0062d10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c0154900 0x7fe8c014cee0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8c00cb390 0x7fe8c0152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 --2- 172.21.15.100:0/92866106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8c00b12d0 0x7fe8c016b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 >> 172.21.15.100:0/92866106 conn(0x7fe8c00678d0 msgr2=0x7fe8c0153180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 shutdown_connections 2024-08-17T02:25:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.064+0000 7fe8c59db700 1 -- 172.21.15.100:0/92866106 wait complete. 2024-08-17T02:25:10.068 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 16 2024-08-17T02:25:10.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:25:10.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.180+0000 7f542759e700 1 Processor -- start 2024-08-17T02:25:10.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.180+0000 7f542759e700 1 -- start start 2024-08-17T02:25:10.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f542806e620 0x7f54280b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f54280b1630 0x7f5428141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5428141a00 con 0x7f54280b0d20 2024-08-17T02:25:10.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5428140b10 con 0x7f54280b1630 2024-08-17T02:25:10.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5428140df0 con 0x7f542806e620 2024-08-17T02:25:10.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f5426d9d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f54280b1630 0x7f5428141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542659c700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f54280b0d20 0x7f542806df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47398/0 (socket says 172.21.15.100:47398) 2024-08-17T02:25:10.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542659c700 1 -- 172.21.15.100:0/590502764 learned_addr learned my addr 172.21.15.100:0/590502764 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f5425d9b700 1 --2- 172.21.15.100:0/590502764 >> v2:172.21.15.123:3300/0 conn(0x7f542806e620 0x7f54280b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f5426d9d700 1 -- 172.21.15.100:0/590502764 >> v2:172.21.15.123:3300/0 conn(0x7f542806e620 msgr2=0x7f54280b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f5426d9d700 1 --2- 172.21.15.100:0/590502764 >> v2:172.21.15.123:3300/0 conn(0x7f542806e620 0x7f54280b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f5426d9d700 1 -- 172.21.15.100:0/590502764 >> v1:172.21.15.100:6789/0 conn(0x7f54280b0d20 legacy=0x7f542806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f5426d9d700 1 -- 172.21.15.100:0/590502764 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54281410d0 con 0x7f54280b1630 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f5426d9d700 1 --2- 172.21.15.100:0/590502764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54280b1630 0x7f5428141e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f542806c630 tx=0x7f5420009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ae2b01385e0321 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542559a700 1 -- 172.21.15.100:0/590502764 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000e070 con 0x7f54280b1630 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542559a700 1 -- 172.21.15.100:0/590502764 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542000bc30 con 0x7f54280b1630 2024-08-17T02:25:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542559a700 1 -- 172.21.15.100:0/590502764 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420012430 con 0x7f54280b1630 2024-08-17T02:25:10.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 -- 172.21.15.100:0/590502764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54280b1630 msgr2=0x7f5428141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 --2- 172.21.15.100:0/590502764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54280b1630 0x7f5428141e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f542806c630 tx=0x7f5420009780 comp rx=0 tx=0).stop 2024-08-17T02:25:10.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 -- 172.21.15.100:0/590502764 shutdown_connections 2024-08-17T02:25:10.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 --2- 172.21.15.100:0/590502764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54280b1630 0x7f5428141e90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 --2- 172.21.15.100:0/590502764 >> v2:172.21.15.123:3300/0 conn(0x7f542806e620 0x7f54280b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 -- 172.21.15.100:0/590502764 >> 172.21.15.100:0/590502764 conn(0x7f54280678d0 msgr2=0x7f5428067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:10.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 -- 172.21.15.100:0/590502764 shutdown_connections 2024-08-17T02:25:10.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.184+0000 7f542759e700 1 -- 172.21.15.100:0/590502764 wait complete. 2024-08-17T02:25:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.188+0000 7f542759e700 1 Processor -- start 2024-08-17T02:25:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.188+0000 7f542759e700 1 -- start start 2024-08-17T02:25:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542806e620 0x7f542813f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54280b0d20 0x7f54280a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54280b1630 0x7f54280a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542759e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280a9780 con 0x7f542806e620 2024-08-17T02:25:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280a6360 con 0x7f54280b0d20 2024-08-17T02:25:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54280a6ee0 con 0x7f54280b1630 2024-08-17T02:25:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542806e620 0x7f542813f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542806e620 0x7f542813f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53626/0 (socket says 172.21.15.100:53626) 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 -- 172.21.15.100:0/4209154745 learned_addr learned my addr 172.21.15.100:0/4209154745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f5425d9b700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54280b0d20 0x7f54280a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f5426d9d700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54280b1630 0x7f54280a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 -- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54280b1630 msgr2=0x7f54280a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54280b1630 0x7f54280a9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 -- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54280b0d20 msgr2=0x7f54280a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54280b0d20 0x7f54280a8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 -- 172.21.15.100:0/4209154745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5420009a20 con 0x7f542806e620 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542659c700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542806e620 0x7f542813f7c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f541800c5c0 tx=0x7f5418007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f541e7fc700 1 -- 172.21.15.100:0/4209154745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418005b30 con 0x7f542806e620 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f541e7fc700 1 -- 172.21.15.100:0/4209154745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418005cb0 con 0x7f542806e620 2024-08-17T02:25:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54280a65b0 con 0x7f542806e620 2024-08-17T02:25:10.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f541e7fc700 1 -- 172.21.15.100:0/4209154745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418007320 con 0x7f542806e620 2024-08-17T02:25:10.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.192+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54280a6b70 con 0x7f542806e620 2024-08-17T02:25:10.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.196+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542806f5f0 con 0x7f542806e620 2024-08-17T02:25:10.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.196+0000 7f541e7fc700 1 -- 172.21.15.100:0/4209154745 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f541801b3b0 con 0x7f542806e620 2024-08-17T02:25:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.196+0000 7f541e7fc700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5408060860 0x7f5408062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.196+0000 7f541e7fc700 1 -- 172.21.15.100:0/4209154745 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f541807c1e0 con 0x7f542806e620 2024-08-17T02:25:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.196+0000 7f5425d9b700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5408060860 0x7f5408062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.196+0000 7f5425d9b700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5408060860 0x7f5408062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5428160930 tx=0x7f5410008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.204+0000 7f541e7fc700 1 -- 172.21.15.100:0/4209154745 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f541804c2b0 con 0x7f542806e620 2024-08-17T02:25:10.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.372+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f542806ebc0 con 0x7f542806e620 2024-08-17T02:25:10.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.372+0000 7f541e7fc700 1 -- 172.21.15.100:0/4209154745 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18647 (secure 0 0 0) 0x7f541804f8d0 con 0x7f542806e620 2024-08-17T02:25:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:10.376 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":38,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:23:08.062746+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:50.544046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:51.275880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000351990.meta","create_time":"2024-08-17T02:23:03.999505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000351990"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000351990.data","create_time":"2024-08-17T02:23:04.098167+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000351990"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:25:10.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5408060860 msgr2=0x7f5408062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5408060860 0x7f5408062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5428160930 tx=0x7f5410008040 comp rx=0 tx=0).stop 2024-08-17T02:25:10.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542806e620 msgr2=0x7f542813f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542806e620 0x7f542813f7c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f541800c5c0 tx=0x7f5418007f90 comp rx=0 tx=0).stop 2024-08-17T02:25:10.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 shutdown_connections 2024-08-17T02:25:10.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5408060860 0x7f5408062d10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54280b1630 0x7f54280a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54280b0d20 0x7f54280a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 --2- 172.21.15.100:0/4209154745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542806e620 0x7f542813f7c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 >> 172.21.15.100:0/4209154745 conn(0x7f54280678d0 msgr2=0x7f542815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:10.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 shutdown_connections 2024-08-17T02:25:10.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.376+0000 7f542759e700 1 -- 172.21.15.100:0/4209154745 wait complete. 2024-08-17T02:25:10.396 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-17T02:25:10.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:25:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.500+0000 7f89bade0700 1 Processor -- start 2024-08-17T02:25:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.500+0000 7f89bade0700 1 -- start start 2024-08-17T02:25:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89b4155d20 0x7f89b40b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40af2b0 con 0x7f89b40ad7c0 2024-08-17T02:25:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40af590 con 0x7f89b406df10 2024-08-17T02:25:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40adc10 con 0x7f89b4155d20 2024-08-17T02:25:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b95dd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f89b40ad7c0 0x7f89b40aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47408/0 (socket says 172.21.15.100:47408) 2024-08-17T02:25:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b95dd700 1 -- 172.21.15.100:0/2303717006 learned_addr learned my addr 172.21.15.100:0/2303717006 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b9dde700 1 --2- 172.21.15.100:0/2303717006 >> v2:172.21.15.123:3300/0 conn(0x7f89b4155d20 0x7f89b40b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b8ddc700 1 -- 172.21.15.100:0/2303717006 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768677141 0 0) 0x7f89b40af2b0 con 0x7f89b40ad7c0 2024-08-17T02:25:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b8ddc700 1 -- 172.21.15.100:0/2303717006 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a0003570 con 0x7f89b40ad7c0 2024-08-17T02:25:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b9dde700 1 -- 172.21.15.100:0/2303717006 >> v1:172.21.15.123:6789/0 conn(0x7f89b406df10 legacy=0x7f89b406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b9dde700 1 -- 172.21.15.100:0/2303717006 >> v1:172.21.15.100:6789/0 conn(0x7f89b40ad7c0 legacy=0x7f89b40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b9dde700 1 -- 172.21.15.100:0/2303717006 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40af870 con 0x7f89b4155d20 2024-08-17T02:25:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b9dde700 1 --2- 172.21.15.100:0/2303717006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b4155d20 0x7f89b40b0350 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f89b00098e0 tx=0x7f89b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf96acb81bf37a9c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b8ddc700 1 -- 172.21.15.100:0/2303717006 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000b640 con 0x7f89b4155d20 2024-08-17T02:25:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b8ddc700 1 -- 172.21.15.100:0/2303717006 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b000b7a0 con 0x7f89b4155d20 2024-08-17T02:25:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89b8ddc700 1 -- 172.21.15.100:0/2303717006 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000bb20 con 0x7f89b4155d20 2024-08-17T02:25:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 -- 172.21.15.100:0/2303717006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b4155d20 msgr2=0x7f89b40b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 --2- 172.21.15.100:0/2303717006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b4155d20 0x7f89b40b0350 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f89b00098e0 tx=0x7f89b00091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 -- 172.21.15.100:0/2303717006 shutdown_connections 2024-08-17T02:25:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 --2- 172.21.15.100:0/2303717006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b4155d20 0x7f89b40b0350 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 -- 172.21.15.100:0/2303717006 >> 172.21.15.100:0/2303717006 conn(0x7f89b40678d0 msgr2=0x7f89b4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 -- 172.21.15.100:0/2303717006 shutdown_connections 2024-08-17T02:25:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.504+0000 7f89bade0700 1 -- 172.21.15.100:0/2303717006 wait complete. 2024-08-17T02:25:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89bade0700 1 Processor -- start 2024-08-17T02:25:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89bade0700 1 -- start start 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89bade0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406df10 0x7f89b4140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89bade0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b40ad7c0 0x7f89b4153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89bade0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89b4155d20 0x7f89b416b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89bade0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4150350 con 0x7f89b406df10 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89bade0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4150630 con 0x7f89b40ad7c0 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89bade0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4150910 con 0x7f89b4155d20 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b9dde700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406df10 0x7f89b4140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b9dde700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406df10 0x7f89b4140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53632/0 (socket says 172.21.15.100:53632) 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b9dde700 1 -- 172.21.15.100:0/2166282889 learned_addr learned my addr 172.21.15.100:0/2166282889 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b95dd700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b40ad7c0 0x7f89b4153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89ba5df700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89b4155d20 0x7f89b416b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b95dd700 1 -- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89b4155d20 msgr2=0x7f89b416b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b95dd700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89b4155d20 0x7f89b416b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b95dd700 1 -- 172.21.15.100:0/2166282889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406df10 msgr2=0x7f89b4140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b95dd700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406df10 0x7f89b4140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b95dd700 1 -- 172.21.15.100:0/2166282889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b0009420 con 0x7f89b40ad7c0 2024-08-17T02:25:10.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b9dde700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406df10 0x7f89b4140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:10.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89ba5df700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89b4155d20 0x7f89b416b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:10.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.508+0000 7f89b95dd700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b40ad7c0 0x7f89b4153850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f89a400c580 tx=0x7f89a4007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89a9ffb700 1 -- 172.21.15.100:0/2166282889 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4005c00 con 0x7f89b40ad7c0 2024-08-17T02:25:10.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89a9ffb700 1 -- 172.21.15.100:0/2166282889 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4005d80 con 0x7f89b40ad7c0 2024-08-17T02:25:10.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b4150bf0 con 0x7f89b40ad7c0 2024-08-17T02:25:10.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89a9ffb700 1 -- 172.21.15.100:0/2166282889 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4007320 con 0x7f89b40ad7c0 2024-08-17T02:25:10.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b4153d90 con 0x7f89b40ad7c0 2024-08-17T02:25:10.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b41510f0 con 0x7f89b40ad7c0 2024-08-17T02:25:10.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89a9ffb700 1 -- 172.21.15.100:0/2166282889 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f89a401b3b0 con 0x7f89b40ad7c0 2024-08-17T02:25:10.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89a9ffb700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8998060860 0x7f8998062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.512+0000 7f89a9ffb700 1 -- 172.21.15.100:0/2166282889 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f89a407c7d0 con 0x7f89b40ad7c0 2024-08-17T02:25:10.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.516+0000 7f89b9dde700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8998060860 0x7f8998062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.516+0000 7f89b9dde700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8998060860 0x7f8998062d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c010 tx=0x7f89b0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.524+0000 7f89a9ffb700 1 -- 172.21.15.100:0/2166282889 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f89a404c920 con 0x7f89b40ad7c0 2024-08-17T02:25:10.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.720+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89b40643a0 con 0x7f89b40ad7c0 2024-08-17T02:25:10.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.720+0000 7f89a9ffb700 1 -- 172.21.15.100:0/2166282889 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5696 (secure 0 0 0) 0x7f89a4017090 con 0x7f89b40ad7c0 2024-08-17T02:25:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:10.724 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:52.296101+0000","modified":"2024-08-17T02:22:54.942750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4980":{"gid":4980,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3037167008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3037167008},{"type":"v1","addr":"172.21.15.100:6835","nonce":3037167008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:23:05.071644+0000","modified":"2024-08-17T02:23:06.938110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000351990","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:25:10.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8998060860 msgr2=0x7f8998062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8998060860 0x7f8998062d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c010 tx=0x7f89b0017000 comp rx=0 tx=0).stop 2024-08-17T02:25:10.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b40ad7c0 msgr2=0x7f89b4153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b40ad7c0 0x7f89b4153850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f89a400c580 tx=0x7f89a4007d50 comp rx=0 tx=0).stop 2024-08-17T02:25:10.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 shutdown_connections 2024-08-17T02:25:10.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8998060860 0x7f8998062d10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89b4155d20 0x7f89b416b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89b40ad7c0 0x7f89b4153850 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 --2- 172.21.15.100:0/2166282889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406df10 0x7f89b4140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 >> 172.21.15.100:0/2166282889 conn(0x7f89b40678d0 msgr2=0x7f89b414c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:10.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 shutdown_connections 2024-08-17T02:25:10.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.724+0000 7f89bade0700 1 -- 172.21.15.100:0/2166282889 wait complete. 2024-08-17T02:25:10.727 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 16 2024-08-17T02:25:10.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:25:10.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.840+0000 7ffba08ab700 1 Processor -- start 2024-08-17T02:25:10.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.840+0000 7ffba08ab700 1 -- start start 2024-08-17T02:25:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffba08ab700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb9c142780 0x7ffb9c0ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffba08ab700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9c0ca510 con 0x7ffb9c142b50 2024-08-17T02:25:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffba08ab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9c0c9900 con 0x7ffb9c0aef00 2024-08-17T02:25:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffba08ab700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c0c9be0 con 0x7ffb9c142780 2024-08-17T02:25:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9b7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb9c142780 0x7ffb9c0ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9affd700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ffb9c142b50 0x7ffb9c0ca910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47410/0 (socket says 172.21.15.100:47410) 2024-08-17T02:25:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9affd700 1 -- 172.21.15.100:0/2957236556 learned_addr learned my addr 172.21.15.100:0/2957236556 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:10.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9a7fc700 1 -- 172.21.15.100:0/2957236556 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923987199 0 0) 0x7ffb9c0ca510 con 0x7ffb9c142b50 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9a7fc700 1 -- 172.21.15.100:0/2957236556 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb84003570 con 0x7ffb9c142b50 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9a7fc700 1 -- 172.21.15.100:0/2957236556 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370477159 0 0) 0x7ffb9c0c9900 con 0x7ffb9c0aef00 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9b7fe700 1 -- 172.21.15.100:0/2957236556 >> v1:172.21.15.123:6789/0 conn(0x7ffb9c0aef00 legacy=0x7ffb9c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9b7fe700 1 -- 172.21.15.100:0/2957236556 >> v1:172.21.15.100:6789/0 conn(0x7ffb9c142b50 legacy=0x7ffb9c0ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9b7fe700 1 -- 172.21.15.100:0/2957236556 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c0c9ec0 con 0x7ffb9c142780 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9b7fe700 1 --2- 172.21.15.100:0/2957236556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142780 0x7ffb9c0ae9c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0098e0 tx=0x7ffb8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a27a50cd90936d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9a7fc700 1 -- 172.21.15.100:0/2957236556 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00b640 con 0x7ffb9c142780 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9a7fc700 1 -- 172.21.15.100:0/2957236556 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c00b7a0 con 0x7ffb9c142780 2024-08-17T02:25:10.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffb9a7fc700 1 -- 172.21.15.100:0/2957236556 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00bb40 con 0x7ffb9c142780 2024-08-17T02:25:10.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffba08ab700 1 -- 172.21.15.100:0/2957236556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142780 msgr2=0x7ffb9c0ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.844+0000 7ffba08ab700 1 --2- 172.21.15.100:0/2957236556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142780 0x7ffb9c0ae9c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0098e0 tx=0x7ffb8c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 -- 172.21.15.100:0/2957236556 shutdown_connections 2024-08-17T02:25:10.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 --2- 172.21.15.100:0/2957236556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142780 0x7ffb9c0ae9c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 -- 172.21.15.100:0/2957236556 >> 172.21.15.100:0/2957236556 conn(0x7ffb9c06e070 msgr2=0x7ffb9c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:10.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 -- 172.21.15.100:0/2957236556 shutdown_connections 2024-08-17T02:25:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 -- 172.21.15.100:0/2957236556 wait complete. 2024-08-17T02:25:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 Processor -- start 2024-08-17T02:25:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 -- start start 2024-08-17T02:25:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb9c0aef00 0x7ffb9c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9c142780 0x7ffb9c12f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142b50 0x7ffb9c12da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c12f3e0 con 0x7ffb9c142b50 2024-08-17T02:25:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c158eb0 con 0x7ffb9c142780 2024-08-17T02:25:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffba08ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c12e7d0 con 0x7ffb9c0aef00 2024-08-17T02:25:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffb9bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142b50 0x7ffb9c12da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffb9bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142b50 0x7ffb9c12da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53648/0 (socket says 172.21.15.100:53648) 2024-08-17T02:25:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffb9bfff700 1 -- 172.21.15.100:0/1317750237 learned_addr learned my addr 172.21.15.100:0/1317750237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffb9affd700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9c142780 0x7ffb9c12f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.848+0000 7ffb9b7fe700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb9c0aef00 0x7ffb9c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb9bfff700 1 -- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb9c0aef00 msgr2=0x7ffb9c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb9bfff700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb9c0aef00 0x7ffb9c158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb9bfff700 1 -- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9c142780 msgr2=0x7ffb9c12f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb9bfff700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9c142780 0x7ffb9c12f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb9bfff700 1 -- 172.21.15.100:0/1317750237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c009420 con 0x7ffb9c142b50 2024-08-17T02:25:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb9bfff700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142b50 0x7ffb9c12da30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffb94002230 tx=0x7ffb940016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb837fe700 1 -- 172.21.15.100:0/1317750237 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94012630 con 0x7ffb9c142b50 2024-08-17T02:25:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb837fe700 1 -- 172.21.15.100:0/1317750237 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94012790 con 0x7ffb9c142b50 2024-08-17T02:25:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffb837fe700 1 -- 172.21.15.100:0/1317750237 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94012b50 con 0x7ffb9c142b50 2024-08-17T02:25:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c12eab0 con 0x7ffb9c142b50 2024-08-17T02:25:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.852+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c12f070 con 0x7ffb9c142b50 2024-08-17T02:25:10.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.856+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c12f550 con 0x7ffb9c142b50 2024-08-17T02:25:10.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.856+0000 7ffb837fe700 1 -- 172.21.15.100:0/1317750237 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ffb9401a3a0 con 0x7ffb9c142b50 2024-08-17T02:25:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.856+0000 7ffb837fe700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffb88060890 0x7ffb88062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.856+0000 7ffb837fe700 1 -- 172.21.15.100:0/1317750237 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7ffb9407d0c0 con 0x7ffb9c142b50 2024-08-17T02:25:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.856+0000 7ffb9b7fe700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffb88060890 0x7ffb88062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.856+0000 7ffb9b7fe700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffb88060890 0x7ffb88062d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0093f0 tx=0x7ffb8c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:10.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:10.864+0000 7ffb837fe700 1 -- 172.21.15.100:0/1317750237 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ffb9404d190 con 0x7ffb9c142b50 2024-08-17T02:25:11.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.064+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb9c0638f0 con 0x7ffb9c142b50 2024-08-17T02:25:11.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.064+0000 7ffb837fe700 1 -- 172.21.15.100:0/1317750237 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5696 (secure 0 0 0) 0x7ffb940507b0 con 0x7ffb9c142b50 2024-08-17T02:25:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:11.067 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:22:52.296101+0000","modified":"2024-08-17T02:22:54.942750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4980":{"gid":4980,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3037167008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3037167008},{"type":"v1","addr":"172.21.15.100:6835","nonce":3037167008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:23:05.071644+0000","modified":"2024-08-17T02:23:06.938110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000351990","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:25:11.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffb88060890 msgr2=0x7ffb88062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffb88060890 0x7ffb88062d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0093f0 tx=0x7ffb8c017000 comp rx=0 tx=0).stop 2024-08-17T02:25:11.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142b50 msgr2=0x7ffb9c12da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142b50 0x7ffb9c12da30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffb94002230 tx=0x7ffb940016f0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 shutdown_connections 2024-08-17T02:25:11.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ffb88060890 0x7ffb88062d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb9c142b50 0x7ffb9c12da30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9c142780 0x7ffb9c12f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 --2- 172.21.15.100:0/1317750237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb9c0aef00 0x7ffb9c158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 >> 172.21.15.100:0/1317750237 conn(0x7ffb9c06e070 msgr2=0x7ffb9c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:11.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 shutdown_connections 2024-08-17T02:25:11.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.068+0000 7ffba08ab700 1 -- 172.21.15.100:0/1317750237 wait complete. 2024-08-17T02:25:11.071 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 16 2024-08-17T02:25:11.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:25:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b7fff700 1 Processor -- start 2024-08-17T02:25:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b7fff700 1 -- start start 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b7fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f88b8159040 0x7f88b80cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b7fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f88b80d0010 0x7f88b80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b7fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b80cf190 con 0x7f88b815cea0 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b7fff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b815d9f0 con 0x7f88b80d0010 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b7fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b80d09a0 con 0x7f88b8159040 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b67fc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f88b80d0010 0x7f88b80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b77fe700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f88b815cea0 0x7f88b815d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47412/0 (socket says 172.21.15.100:47412) 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b77fe700 1 -- 172.21.15.100:0/1772235077 learned_addr learned my addr 172.21.15.100:0/1772235077 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b6ffd700 1 --2- 172.21.15.100:0/1772235077 >> v2:172.21.15.123:3300/0 conn(0x7f88b8159040 0x7f88b80cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b5ffb700 1 -- 172.21.15.100:0/1772235077 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810818156 0 0) 0x7f88b80cf190 con 0x7f88b815cea0 2024-08-17T02:25:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b5ffb700 1 -- 172.21.15.100:0/1772235077 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a0003570 con 0x7f88b815cea0 2024-08-17T02:25:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b67fc700 1 -- 172.21.15.100:0/1772235077 >> v2:172.21.15.123:3300/0 conn(0x7f88b8159040 msgr2=0x7f88b80cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b67fc700 1 --2- 172.21.15.100:0/1772235077 >> v2:172.21.15.123:3300/0 conn(0x7f88b8159040 0x7f88b80cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b67fc700 1 -- 172.21.15.100:0/1772235077 >> v1:172.21.15.100:6789/0 conn(0x7f88b815cea0 legacy=0x7f88b815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b67fc700 1 -- 172.21.15.100:0/1772235077 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80ce5e0 con 0x7f88b80d0010 2024-08-17T02:25:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b67fc700 1 --2- 172.21.15.100:0/1772235077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b80d0010 0x7f88b80d0460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f88b8070d70 tx=0x7f88ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69fa29bfa5ea434 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b5ffb700 1 -- 172.21.15.100:0/1772235077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac011630 con 0x7f88b80d0010 2024-08-17T02:25:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b5ffb700 1 -- 172.21.15.100:0/1772235077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac0117b0 con 0x7f88b80d0010 2024-08-17T02:25:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.180+0000 7f88b5ffb700 1 -- 172.21.15.100:0/1772235077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac011b70 con 0x7f88b80d0010 2024-08-17T02:25:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 -- 172.21.15.100:0/1772235077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b80d0010 msgr2=0x7f88b80d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 --2- 172.21.15.100:0/1772235077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b80d0010 0x7f88b80d0460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f88b8070d70 tx=0x7f88ac0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 -- 172.21.15.100:0/1772235077 shutdown_connections 2024-08-17T02:25:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 --2- 172.21.15.100:0/1772235077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b80d0010 0x7f88b80d0460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 --2- 172.21.15.100:0/1772235077 >> v2:172.21.15.123:3300/0 conn(0x7f88b8159040 0x7f88b80cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 -- 172.21.15.100:0/1772235077 >> 172.21.15.100:0/1772235077 conn(0x7f88b8068250 msgr2=0x7f88b8068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 -- 172.21.15.100:0/1772235077 shutdown_connections 2024-08-17T02:25:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 -- 172.21.15.100:0/1772235077 wait complete. 2024-08-17T02:25:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 Processor -- start 2024-08-17T02:25:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.184+0000 7f88b7fff700 1 -- start start 2024-08-17T02:25:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88b80d0010 0x7f88b815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88b8159040 0x7f88b815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b815cea0 0x7f88b815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8147400 con 0x7f88b815cea0 2024-08-17T02:25:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b815cba0 con 0x7f88b80d0010 2024-08-17T02:25:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8151ff0 con 0x7f88b8159040 2024-08-17T02:25:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b815cea0 0x7f88b815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b815cea0 0x7f88b815c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53664/0 (socket says 172.21.15.100:53664) 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 -- 172.21.15.100:0/2303578374 learned_addr learned my addr 172.21.15.100:0/2303578374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b6ffd700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88b80d0010 0x7f88b815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b67fc700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88b8159040 0x7f88b815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 -- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88b8159040 msgr2=0x7f88b815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88b8159040 0x7f88b815c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 -- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88b80d0010 msgr2=0x7f88b815a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88b80d0010 0x7f88b815a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 -- 172.21.15.100:0/2303578374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac009420 con 0x7f88b815cea0 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b77fe700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b815cea0 0x7f88b815c660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f88b0001460 tx=0x7f88b0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f889effd700 1 -- 172.21.15.100:0/2303578374 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b000a890 con 0x7f88b815cea0 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f889effd700 1 -- 172.21.15.100:0/2303578374 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b000a9f0 con 0x7f88b815cea0 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b81522d0 con 0x7f88b815cea0 2024-08-17T02:25:11.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f889effd700 1 -- 172.21.15.100:0/2303578374 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b000ad70 con 0x7f88b815cea0 2024-08-17T02:25:11.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b8152890 con 0x7f88b815cea0 2024-08-17T02:25:11.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.188+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b80b0660 con 0x7f88b815cea0 2024-08-17T02:25:11.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.192+0000 7f889effd700 1 -- 172.21.15.100:0/2303578374 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f88b0007300 con 0x7f88b815cea0 2024-08-17T02:25:11.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.192+0000 7f889effd700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f88a4060860 0x7f88a4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.192+0000 7f889effd700 1 -- 172.21.15.100:0/2303578374 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f88b007d2e0 con 0x7f88b815cea0 2024-08-17T02:25:11.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.192+0000 7f88b6ffd700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f88a4060860 0x7f88a4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.192+0000 7f88b6ffd700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f88a4060860 0x7f88a4062d10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009d10 tx=0x7f88a80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.200+0000 7f889effd700 1 -- 172.21.15.100:0/2303578374 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f88b80b0660 con 0x7f88b815cea0 2024-08-17T02:25:11.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.368+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88b80b0660 con 0x7f88b815cea0 2024-08-17T02:25:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f889effd700 1 -- 172.21.15.100:0/2303578374 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18647 (secure 0 0 0) 0x7f88b80b0660 con 0x7f88b815cea0 2024-08-17T02:25:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:11.373 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":38,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:23:08.062746+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:22:50.544046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:22:51.275880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000351990.meta","create_time":"2024-08-17T02:23:03.999505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000351990"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000351990.data","create_time":"2024-08-17T02:23:04.098167+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000351990"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f88a4060860 msgr2=0x7f88a4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f88a4060860 0x7f88a4062d10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009d10 tx=0x7f88a80091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b815cea0 msgr2=0x7f88b815c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b815cea0 0x7f88b815c660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f88b0001460 tx=0x7f88b0001ae0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 shutdown_connections 2024-08-17T02:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f88a4060860 0x7f88a4062d10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b815cea0 0x7f88b815c660 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88b8159040 0x7f88b815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 --2- 172.21.15.100:0/2303578374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88b80d0010 0x7f88b815a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 >> 172.21.15.100:0/2303578374 conn(0x7f88b8068250 msgr2=0x7f88b8154970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 shutdown_connections 2024-08-17T02:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.372+0000 7f88b7fff700 1 -- 172.21.15.100:0/2303578374 wait complete. 2024-08-17T02:25:11.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-17T02:25:11.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.488+0000 7fdb188af700 1 Processor -- start 2024-08-17T02:25:11.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb188af700 1 -- start start 2024-08-17T02:25:11.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb188af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb14153e40 0x7fdb1406e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb188af700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdb1406e7f0 0x7fdb14158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb188af700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb14157810 con 0x7fdb14153a70 2024-08-17T02:25:11.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb188af700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb14157dd0 con 0x7fdb1406e7f0 2024-08-17T02:25:11.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb188af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1406ec40 con 0x7fdb14153e40 2024-08-17T02:25:11.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb11d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb14153e40 0x7fdb1406e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb12d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdb1406e7f0 0x7fdb14158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb11d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb14153e40 0x7fdb1406e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:43362/0 (socket says 172.21.15.100:43362) 2024-08-17T02:25:11.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb11d9b700 1 -- 172.21.15.100:0/1802352189 learned_addr learned my addr 172.21.15.100:0/1802352189 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:11.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb12d9d700 1 -- 172.21.15.100:0/1802352189 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb14153e40 msgr2=0x7fdb1406e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb12d9d700 1 --2- 172.21.15.100:0/1802352189 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb14153e40 0x7fdb1406e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb12d9d700 1 -- 172.21.15.100:0/1802352189 >> v1:172.21.15.123:6789/0 conn(0x7fdb14153a70 legacy=0x7fdb14159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb12d9d700 1 -- 172.21.15.100:0/1802352189 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1415a700 con 0x7fdb1406e7f0 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb12d9d700 1 --2- 172.21.15.100:0/1802352189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb1406e7f0 0x7fdb14158260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009e60 tx=0x7fdb0c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c24df1f00f6cf3 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb1159a700 1 -- 172.21.15.100:0/1802352189 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00bb60 con 0x7fdb1406e7f0 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb1159a700 1 -- 172.21.15.100:0/1802352189 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c00bcc0 con 0x7fdb1406e7f0 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.492+0000 7fdb1159a700 1 -- 172.21.15.100:0/1802352189 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0116b0 con 0x7fdb1406e7f0 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 -- 172.21.15.100:0/1802352189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb1406e7f0 msgr2=0x7fdb14158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 --2- 172.21.15.100:0/1802352189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb1406e7f0 0x7fdb14158260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009e60 tx=0x7fdb0c009780 comp rx=0 tx=0).stop 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 -- 172.21.15.100:0/1802352189 shutdown_connections 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 --2- 172.21.15.100:0/1802352189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb1406e7f0 0x7fdb14158260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 --2- 172.21.15.100:0/1802352189 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb14153e40 0x7fdb1406e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 -- 172.21.15.100:0/1802352189 >> 172.21.15.100:0/1802352189 conn(0x7fdb14067810 msgr2=0x7fdb14067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:11.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 -- 172.21.15.100:0/1802352189 shutdown_connections 2024-08-17T02:25:11.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 -- 172.21.15.100:0/1802352189 wait complete. 2024-08-17T02:25:11.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 Processor -- start 2024-08-17T02:25:11.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.496+0000 7fdb188af700 1 -- start start 2024-08-17T02:25:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb1406e7f0 0x7fdb140cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb14153a70 0x7fdb140afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb14153e40 0x7fdb140b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1414ea60 con 0x7fdb14153e40 2024-08-17T02:25:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140b07e0 con 0x7fdb14153a70 2024-08-17T02:25:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140b0ac0 con 0x7fdb1406e7f0 2024-08-17T02:25:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb12d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb14153e40 0x7fdb140b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb11d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb14153a70 0x7fdb140afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb12d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb14153e40 0x7fdb140b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53676/0 (socket says 172.21.15.100:53676) 2024-08-17T02:25:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb12d9d700 1 -- 172.21.15.100:0/1099020760 learned_addr learned my addr 172.21.15.100:0/1099020760 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb1259c700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb1406e7f0 0x7fdb140cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb11d9b700 1 -- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb1406e7f0 msgr2=0x7fdb140cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb11d9b700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb1406e7f0 0x7fdb140cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb11d9b700 1 -- 172.21.15.100:0/1099020760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb14153e40 msgr2=0x7fdb140b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb11d9b700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb14153e40 0x7fdb140b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb11d9b700 1 -- 172.21.15.100:0/1099020760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c009a20 con 0x7fdb14153a70 2024-08-17T02:25:11.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb11d9b700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb14153a70 0x7fdb140afc20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006fd0 tx=0x7fdafc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb0a7fc700 1 -- 172.21.15.100:0/1099020760 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc013680 con 0x7fdb14153a70 2024-08-17T02:25:11.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb0a7fc700 1 -- 172.21.15.100:0/1099020760 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc0137e0 con 0x7fdb14153a70 2024-08-17T02:25:11.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb140ab020 con 0x7fdb14153a70 2024-08-17T02:25:11.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb0a7fc700 1 -- 172.21.15.100:0/1099020760 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc013b60 con 0x7fdb14153a70 2024-08-17T02:25:11.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb140cb5b0 con 0x7fdb14153a70 2024-08-17T02:25:11.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.500+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb140b0c80 con 0x7fdb14153a70 2024-08-17T02:25:11.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.504+0000 7fdb0a7fc700 1 -- 172.21.15.100:0/1099020760 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdafc026430 con 0x7fdb14153a70 2024-08-17T02:25:11.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.504+0000 7fdb0a7fc700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdaf4060860 0x7fdaf4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.504+0000 7fdb0a7fc700 1 -- 172.21.15.100:0/1099020760 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fdafc07d650 con 0x7fdb14153a70 2024-08-17T02:25:11.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.504+0000 7fdb1259c700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdaf4060860 0x7fdaf4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.504+0000 7fdb1259c700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdaf4060860 0x7fdaf4062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb04003a80 tx=0x7fdb04007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.512+0000 7fdb0a7fc700 1 -- 172.21.15.100:0/1099020760 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdafc04d7a0 con 0x7fdb14153a70 2024-08-17T02:25:11.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.716+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdb140ab520 con 0x7fdb14153a70 2024-08-17T02:25:11.721 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-08-17T02:25:11.720+0000 7f6fd553c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-17T02:25:11.730 INFO:tasks.ceph.mds.a.smithi100.stderr: -7198> 2024-08-17T02:22:56.627+0000 7fcf7c5e2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:25:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.732+0000 7fdb0a7fc700 1 -- 172.21.15.100:0/1099020760 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdafc050dc0 con 0x7fdb14153a70 2024-08-17T02:25:11.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdaf4060860 msgr2=0x7fdaf4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdaf4060860 0x7fdaf4062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb04003a80 tx=0x7fdb04007ec0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb14153a70 msgr2=0x7fdb140afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb14153a70 0x7fdb140afc20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006fd0 tx=0x7fdafc009250 comp rx=0 tx=0).stop 2024-08-17T02:25:11.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 shutdown_connections 2024-08-17T02:25:11.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fdaf4060860 0x7fdaf4062d10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb14153e40 0x7fdb140b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb14153a70 0x7fdb140afc20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 --2- 172.21.15.100:0/1099020760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb1406e7f0 0x7fdb140cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 >> 172.21.15.100:0/1099020760 conn(0x7fdb14067810 msgr2=0x7fdb1406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:11.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 shutdown_connections 2024-08-17T02:25:11.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.736+0000 7fdb188af700 1 -- 172.21.15.100:0/1099020760 wait complete. 2024-08-17T02:25:11.740 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-17T02:25:11.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-17T02:25:11.757 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-08-17T02:25:11.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.848+0000 7f3b52b69700 1 Processor -- start 2024-08-17T02:25:11.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.848+0000 7f3b52b69700 1 -- start start 2024-08-17T02:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3b4c13f680 0x7f3b4c0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c06e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4c0b0810 con 0x7f3b4c13fa50 2024-08-17T02:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4c0afc70 con 0x7f3b4c13f680 2024-08-17T02:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0af060 con 0x7f3b4c0ad6c0 2024-08-17T02:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b51b67700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3b4c13f680 0x7f3b4c0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b51b67700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3b4c13f680 0x7f3b4c0aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53684/0 (socket says 172.21.15.100:53684) 2024-08-17T02:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b51b67700 1 -- 172.21.15.100:0/2536187143 learned_addr learned my addr 172.21.15.100:0/2536187143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52368700 1 --2- 172.21.15.100:0/2536187143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c06e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b51b67700 1 -- 172.21.15.100:0/2536187143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 msgr2=0x7f3b4c06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b51b67700 1 --2- 172.21.15.100:0/2536187143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c06e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b51b67700 1 -- 172.21.15.100:0/2536187143 >> v1:172.21.15.100:6789/0 conn(0x7f3b4c13fa50 legacy=0x7f3b4c0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b51b67700 1 -- 172.21.15.100:0/2536187143 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c0af340 con 0x7f3b4c13f680 2024-08-17T02:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b51b67700 1 --2- 172.21.15.100:0/2536187143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13f680 0x7f3b4c0aea80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3b480099b0 tx=0x7f3b48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a730a01b35fed17d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b50b65700 1 -- 172.21.15.100:0/2536187143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4800b860 con 0x7f3b4c13f680 2024-08-17T02:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b50b65700 1 -- 172.21.15.100:0/2536187143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4800b9c0 con 0x7f3b4c13f680 2024-08-17T02:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b50b65700 1 -- 172.21.15.100:0/2536187143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4800bd60 con 0x7f3b4c13f680 2024-08-17T02:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 -- 172.21.15.100:0/2536187143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13f680 msgr2=0x7f3b4c0aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 --2- 172.21.15.100:0/2536187143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13f680 0x7f3b4c0aea80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3b480099b0 tx=0x7f3b48009250 comp rx=0 tx=0).stop 2024-08-17T02:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 -- 172.21.15.100:0/2536187143 shutdown_connections 2024-08-17T02:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 --2- 172.21.15.100:0/2536187143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c06e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 --2- 172.21.15.100:0/2536187143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13f680 0x7f3b4c0aea80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 -- 172.21.15.100:0/2536187143 >> 172.21.15.100:0/2536187143 conn(0x7f3b4c0678a0 msgr2=0x7f3b4c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 -- 172.21.15.100:0/2536187143 shutdown_connections 2024-08-17T02:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.852+0000 7f3b52b69700 1 -- 172.21.15.100:0/2536187143 wait complete. 2024-08-17T02:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 Processor -- start 2024-08-17T02:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 -- start start 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b4c13f680 0x7f3b4c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13fa50 0x7f3b4c133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c156ca0 con 0x7f3b4c13fa50 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1583a0 con 0x7f3b4c13f680 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c158680 con 0x7f3b4c0ad6c0 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52368700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13fa50 0x7f3b4c133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b51b67700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b51b67700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:43408/0 (socket says 172.21.15.100:43408) 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52368700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13fa50 0x7f3b4c133ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53694/0 (socket says 172.21.15.100:53694) 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b51b67700 1 -- 172.21.15.100:0/3588096629 learned_addr learned my addr 172.21.15.100:0/3588096629 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b51366700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b4c13f680 0x7f3b4c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52368700 1 -- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 msgr2=0x7f3b4c156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52368700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52368700 1 -- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b4c13f680 msgr2=0x7f3b4c157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52368700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b4c13f680 0x7f3b4c157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52368700 1 -- 172.21.15.100:0/3588096629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480094f0 con 0x7f3b4c13fa50 2024-08-17T02:25:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52368700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13fa50 0x7f3b4c133ee0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b44007ed0 tx=0x7f3b44009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b41ffb700 1 -- 172.21.15.100:0/3588096629 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44018070 con 0x7f3b4c13fa50 2024-08-17T02:25:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b41ffb700 1 -- 172.21.15.100:0/3588096629 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4400ec20 con 0x7f3b4c13fa50 2024-08-17T02:25:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c158960 con 0x7f3b4c13fa50 2024-08-17T02:25:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b41ffb700 1 -- 172.21.15.100:0/3588096629 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44014420 con 0x7f3b4c13fa50 2024-08-17T02:25:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.856+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c158f20 con 0x7f3b4c13fa50 2024-08-17T02:25:11.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.860+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4c159140 con 0x7f3b4c13fa50 2024-08-17T02:25:11.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.860+0000 7f3b41ffb700 1 -- 172.21.15.100:0/3588096629 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3b44003020 con 0x7f3b4c13fa50 2024-08-17T02:25:11.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.860+0000 7f3b41ffb700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3b30060860 0x7f3b30062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:11.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.860+0000 7f3b41ffb700 1 -- 172.21.15.100:0/3588096629 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f3b4407d180 con 0x7f3b4c13fa50 2024-08-17T02:25:11.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.860+0000 7f3b51b67700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3b30060860 0x7f3b30062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:11.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.864+0000 7f3b51b67700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3b30060860 0x7f3b30062d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3b48006e80 tx=0x7f3b48017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:11.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:11.868+0000 7f3b41ffb700 1 -- 172.21.15.100:0/3588096629 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3b4404d1f0 con 0x7f3b4c13fa50 2024-08-17T02:25:12.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.068+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3b4c064330 con 0x7f3b4c13fa50 2024-08-17T02:25:12.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.728+0000 7f3b41ffb700 1 -- 172.21.15.100:0/3588096629 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3b44050810 con 0x7f3b4c13fa50 2024-08-17T02:25:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3b30060860 msgr2=0x7f3b30062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3b30060860 0x7f3b30062d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3b48006e80 tx=0x7f3b48017000 comp rx=0 tx=0).stop 2024-08-17T02:25:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13fa50 msgr2=0x7f3b4c133ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13fa50 0x7f3b4c133ee0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b44007ed0 tx=0x7f3b44009850 comp rx=0 tx=0).stop 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 shutdown_connections 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3b30060860 0x7f3b30062d10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4c13fa50 0x7f3b4c133ee0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b4c13f680 0x7f3b4c157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 --2- 172.21.15.100:0/3588096629 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b4c0ad6c0 0x7f3b4c156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 >> 172.21.15.100:0/3588096629 conn(0x7f3b4c0678a0 msgr2=0x7f3b4c13dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 shutdown_connections 2024-08-17T02:25:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.732+0000 7f3b52b69700 1 -- 172.21.15.100:0/3588096629 wait complete. 2024-08-17T02:25:12.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-17T02:25:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.860+0000 7f1c70a89700 1 Processor -- start 2024-08-17T02:25:12.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.860+0000 7f1c70a89700 1 -- start start 2024-08-17T02:25:12.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1c6c0c3660 0x7f1c6c0697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:12.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c0c4850 con 0x7f1c6c0c99c0 2024-08-17T02:25:12.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c15df20 con 0x7f1c6c0c3290 2024-08-17T02:25:12.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6c0cdef0 con 0x7f1c6c0c3660 2024-08-17T02:25:12.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c69d9b700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1c6c0c99c0 0x7f1c6c15d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47428/0 (socket says 172.21.15.100:47428) 2024-08-17T02:25:12.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c69d9b700 1 -- 172.21.15.100:0/2945635536 learned_addr learned my addr 172.21.15.100:0/2945635536 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:12.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6ad9d700 1 --2- 172.21.15.100:0/2945635536 >> v2:172.21.15.123:3300/0 conn(0x7f1c6c0c3660 0x7f1c6c0697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6959a700 1 -- 172.21.15.100:0/2945635536 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508934430 0 0) 0x7f1c6c0c4850 con 0x7f1c6c0c99c0 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6959a700 1 -- 172.21.15.100:0/2945635536 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c54003570 con 0x7f1c6c0c99c0 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6ad9d700 1 -- 172.21.15.100:0/2945635536 >> v1:172.21.15.123:6789/0 conn(0x7f1c6c0c3290 legacy=0x7f1c6c15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6ad9d700 1 -- 172.21.15.100:0/2945635536 >> v1:172.21.15.100:6789/0 conn(0x7f1c6c0c99c0 legacy=0x7f1c6c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6ad9d700 1 -- 172.21.15.100:0/2945635536 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0c3c40 con 0x7f1c6c0c3660 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6ad9d700 1 --2- 172.21.15.100:0/2945635536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 0x7f1c6c0697b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009e60 tx=0x7f1c64009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af60334a6c10b41e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6959a700 1 -- 172.21.15.100:0/2945635536 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400e070 con 0x7f1c6c0c3660 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6959a700 1 -- 172.21.15.100:0/2945635536 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6400bb60 con 0x7f1c6c0c3660 2024-08-17T02:25:12.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c6959a700 1 -- 172.21.15.100:0/2945635536 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64012430 con 0x7f1c6c0c3660 2024-08-17T02:25:12.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 -- 172.21.15.100:0/2945635536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 msgr2=0x7f1c6c0697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:12.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 --2- 172.21.15.100:0/2945635536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 0x7f1c6c0697b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009e60 tx=0x7f1c64009780 comp rx=0 tx=0).stop 2024-08-17T02:25:12.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 -- 172.21.15.100:0/2945635536 shutdown_connections 2024-08-17T02:25:12.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 --2- 172.21.15.100:0/2945635536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 0x7f1c6c0697b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:12.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 -- 172.21.15.100:0/2945635536 >> 172.21.15.100:0/2945635536 conn(0x7f1c6c0cab00 msgr2=0x7f1c6c0caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:12.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 -- 172.21.15.100:0/2945635536 shutdown_connections 2024-08-17T02:25:12.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.864+0000 7f1c70a89700 1 -- 172.21.15.100:0/2945635536 wait complete. 2024-08-17T02:25:12.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c70a89700 1 Processor -- start 2024-08-17T02:25:12.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c70a89700 1 -- start start 2024-08-17T02:25:12.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c70a89700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c6c0c3290 0x7f1c6c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c70a89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 0x7f1c6c139360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c70a89700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c6c0c99c0 0x7f1c6c1398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c70a89700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c15ab30 con 0x7f1c6c0c3290 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c70a89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c139de0 con 0x7f1c6c0c3660 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c70a89700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c1593e0 con 0x7f1c6c0c99c0 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c6a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c6c0c3290 0x7f1c6c158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c6a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c6c0c3290 0x7f1c6c158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53698/0 (socket says 172.21.15.100:53698) 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.868+0000 7f1c6a59c700 1 -- 172.21.15.100:0/338740391 learned_addr learned my addr 172.21.15.100:0/338740391 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c69d9b700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 0x7f1c6c139360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c6ad9d700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c6c0c99c0 0x7f1c6c1398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:12.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c6a59c700 1 -- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c6c0c99c0 msgr2=0x7f1c6c1398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c6a59c700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c6c0c99c0 0x7f1c6c1398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c6a59c700 1 -- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 msgr2=0x7f1c6c139360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c6a59c700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 0x7f1c6c139360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c6a59c700 1 -- 172.21.15.100:0/338740391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64009a20 con 0x7f1c6c0c3290 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c6ad9d700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c6c0c99c0 0x7f1c6c1398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c6a59c700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c6c0c3290 0x7f1c6c158ea0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c7c0 tx=0x7f1c60007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c5a7fc700 1 -- 172.21.15.100:0/338740391 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60011590 con 0x7f1c6c0c3290 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c5a7fc700 1 -- 172.21.15.100:0/338740391 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c60011710 con 0x7f1c6c0c3290 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c159f20 con 0x7f1c6c0c3290 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c5a7fc700 1 -- 172.21.15.100:0/338740391 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60011ad0 con 0x7f1c6c0c3290 2024-08-17T02:25:12.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c15a4e0 con 0x7f1c6c0c3290 2024-08-17T02:25:12.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c0638f0 con 0x7f1c6c0c3290 2024-08-17T02:25:12.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.872+0000 7f1c5a7fc700 1 -- 172.21.15.100:0/338740391 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1c6001b3b0 con 0x7f1c6c0c3290 2024-08-17T02:25:12.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.876+0000 7f1c5a7fc700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1c4c060860 0x7f1c4c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:12.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.876+0000 7f1c5a7fc700 1 -- 172.21.15.100:0/338740391 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f1c6007c450 con 0x7f1c6c0c3290 2024-08-17T02:25:12.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.876+0000 7f1c69d9b700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1c4c060860 0x7f1c4c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:12.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.876+0000 7f1c69d9b700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1c4c060860 0x7f1c4c062d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c06c9a0 tx=0x7f1c5c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:12.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:12.884+0000 7f1c5a7fc700 1 -- 172.21.15.100:0/338740391 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1c6004c4c0 con 0x7f1c6c0c3290 2024-08-17T02:25:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.048+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1c6c067ed0 con 0x7f1c6c0c3290 2024-08-17T02:25:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.736+0000 7f1c5a7fc700 1 -- 172.21.15.100:0/338740391 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1c600170d0 con 0x7f1c6c0c3290 2024-08-17T02:25:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1c4c060860 msgr2=0x7f1c4c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1c4c060860 0x7f1c4c062d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c06c9a0 tx=0x7f1c5c009040 comp rx=0 tx=0).stop 2024-08-17T02:25:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c6c0c3290 msgr2=0x7f1c6c158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c6c0c3290 0x7f1c6c158ea0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c7c0 tx=0x7f1c60007f40 comp rx=0 tx=0).stop 2024-08-17T02:25:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 shutdown_connections 2024-08-17T02:25:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1c4c060860 0x7f1c4c062d10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c6c0c99c0 0x7f1c6c1398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c0c3660 0x7f1c6c139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 --2- 172.21.15.100:0/338740391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c6c0c3290 0x7f1c6c158ea0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 >> 172.21.15.100:0/338740391 conn(0x7f1c6c0cab00 msgr2=0x7f1c6c0c88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 shutdown_connections 2024-08-17T02:25:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.744+0000 7f1c70a89700 1 -- 172.21.15.100:0/338740391 wait complete. 2024-08-17T02:25:13.749 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-08-17T02:25:13.761 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-17T02:25:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.876+0000 7ff47019c700 1 Processor -- start 2024-08-17T02:25:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.876+0000 7ff47019c700 1 -- start start 2024-08-17T02:25:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.876+0000 7ff47019c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff468159500 0x7ff4680cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.876+0000 7ff47019c700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680d09c0 con 0x7ff4680cf110 2024-08-17T02:25:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.876+0000 7ff47019c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46815aab0 con 0x7ff4680ca850 2024-08-17T02:25:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.876+0000 7ff47019c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680d0c10 con 0x7ff468159500 2024-08-17T02:25:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.876+0000 7ff46d737700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff4680cf110 0x7ff4680cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47438/0 (socket says 172.21.15.100:47438) 2024-08-17T02:25:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.876+0000 7ff46d737700 1 -- 172.21.15.100:0/324739289 learned_addr learned my addr 172.21.15.100:0/324739289 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46df38700 1 --2- 172.21.15.100:0/324739289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff468159500 0x7ff4680cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46cf36700 1 -- 172.21.15.100:0/324739289 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180599884 0 0) 0x7ff4680d09c0 con 0x7ff4680cf110 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46cf36700 1 -- 172.21.15.100:0/324739289 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff450003570 con 0x7ff4680cf110 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46df38700 1 -- 172.21.15.100:0/324739289 >> v1:172.21.15.123:6789/0 conn(0x7ff4680ca850 legacy=0x7ff4680cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46cf36700 1 -- 172.21.15.100:0/324739289 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192899592 0 0) 0x7ff46815aab0 con 0x7ff4680ca850 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46df38700 1 -- 172.21.15.100:0/324739289 >> v1:172.21.15.100:6789/0 conn(0x7ff4680cf110 legacy=0x7ff4680cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46df38700 1 -- 172.21.15.100:0/324739289 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4680d0ef0 con 0x7ff468159500 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46df38700 1 --2- 172.21.15.100:0/324739289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff468159500 0x7ff4680cebd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff464009e90 tx=0x7ff464009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1451d650c053bce9 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46cf36700 1 -- 172.21.15.100:0/324739289 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400e070 con 0x7ff468159500 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46cf36700 1 -- 172.21.15.100:0/324739289 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46400bc30 con 0x7ff468159500 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff46cf36700 1 -- 172.21.15.100:0/324739289 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464012430 con 0x7ff468159500 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 -- 172.21.15.100:0/324739289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff468159500 msgr2=0x7ff4680cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 --2- 172.21.15.100:0/324739289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff468159500 0x7ff4680cebd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff464009e90 tx=0x7ff464009780 comp rx=0 tx=0).stop 2024-08-17T02:25:13.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 -- 172.21.15.100:0/324739289 shutdown_connections 2024-08-17T02:25:13.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 --2- 172.21.15.100:0/324739289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff468159500 0x7ff4680cebd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:13.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 -- 172.21.15.100:0/324739289 >> 172.21.15.100:0/324739289 conn(0x7ff468067590 msgr2=0x7ff4680d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:13.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 -- 172.21.15.100:0/324739289 shutdown_connections 2024-08-17T02:25:13.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 -- 172.21.15.100:0/324739289 wait complete. 2024-08-17T02:25:13.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 Processor -- start 2024-08-17T02:25:13.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.880+0000 7ff47019c700 1 -- start start 2024-08-17T02:25:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4680ca850 0x7ff468157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4680cf110 0x7ff4680c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff468159500 0x7ff4680b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680b5c10 con 0x7ff4680ca850 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680b5000 con 0x7ff468159500 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680b52e0 con 0x7ff4680cf110 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4680ca850 0x7ff468157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4680ca850 0x7ff468157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53708/0 (socket says 172.21.15.100:53708) 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 -- 172.21.15.100:0/2826527425 learned_addr learned my addr 172.21.15.100:0/2826527425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46d737700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4680cf110 0x7ff4680c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46e739700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff468159500 0x7ff4680b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 -- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4680cf110 msgr2=0x7ff4680c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4680cf110 0x7ff4680c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 -- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff468159500 msgr2=0x7ff4680b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff468159500 0x7ff4680b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 -- 172.21.15.100:0/2826527425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464009a20 con 0x7ff4680ca850 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46e739700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff468159500 0x7ff4680b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff46df38700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4680ca850 0x7ff468157a60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff468088c50 tx=0x7ff464019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff45dffb700 1 -- 172.21.15.100:0/2826527425 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464007060 con 0x7ff4680ca850 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff45dffb700 1 -- 172.21.15.100:0/2826527425 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff464012da0 con 0x7ff4680ca850 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4680b55c0 con 0x7ff4680ca850 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff45dffb700 1 -- 172.21.15.100:0/2826527425 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464026640 con 0x7ff4680ca850 2024-08-17T02:25:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff468157fa0 con 0x7ff4680ca850 2024-08-17T02:25:13.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.884+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4680b5d80 con 0x7ff4680ca850 2024-08-17T02:25:13.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.888+0000 7ff45dffb700 1 -- 172.21.15.100:0/2826527425 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff4640269b0 con 0x7ff4680ca850 2024-08-17T02:25:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.888+0000 7ff45dffb700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff454060890 0x7ff454062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.888+0000 7ff45dffb700 1 -- 172.21.15.100:0/2826527425 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6251+0+0 (secure 0 0 0) 0x7ff46407e960 con 0x7ff4680ca850 2024-08-17T02:25:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.888+0000 7ff46d737700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff454060890 0x7ff454062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.888+0000 7ff46d737700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff454060890 0x7ff454062d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4680cf590 tx=0x7ff460009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:13.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:13.896+0000 7ff45dffb700 1 -- 172.21.15.100:0/2826527425 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff46404eb90 con 0x7ff4680ca850 2024-08-17T02:25:14.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.060+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff4680638f0 con 0x7ff4680ca850 2024-08-17T02:25:14.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.744+0000 7ff45dffb700 1 -- 172.21.15.100:0/2826527425 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff4640521b0 con 0x7ff4680ca850 2024-08-17T02:25:14.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff454060890 msgr2=0x7ff454062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff454060890 0x7ff454062d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4680cf590 tx=0x7ff460009040 comp rx=0 tx=0).stop 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4680ca850 msgr2=0x7ff468157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4680ca850 0x7ff468157a60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff468088c50 tx=0x7ff464019000 comp rx=0 tx=0).stop 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 shutdown_connections 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7ff454060890 0x7ff454062d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff468159500 0x7ff4680b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4680cf110 0x7ff4680c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 --2- 172.21.15.100:0/2826527425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4680ca850 0x7ff468157a60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 >> 172.21.15.100:0/2826527425 conn(0x7ff468067590 msgr2=0x7ff4681583e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:14.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 shutdown_connections 2024-08-17T02:25:14.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.748+0000 7ff47019c700 1 -- 172.21.15.100:0/2826527425 wait complete. 2024-08-17T02:25:14.751 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-08-17T02:25:14.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:25:14.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.860+0000 7fb4eb4a2700 1 Processor -- start 2024-08-17T02:25:14.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.860+0000 7fb4eb4a2700 1 -- start start 2024-08-17T02:25:14.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eb4a2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e40ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:14.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eb4a2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb4e40cae50 0x7fb4e415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:14.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eb4a2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e40ca510 con 0x7fb4e4142780 2024-08-17T02:25:14.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eb4a2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e40cb290 con 0x7fb4e40cae50 2024-08-17T02:25:14.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eb4a2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40c9900 con 0x7fb4e4142b50 2024-08-17T02:25:14.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4e9c9f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e40ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:14.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eaca1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb4e40cae50 0x7fb4e415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:14.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eaca1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb4e40cae50 0x7fb4e415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:53444/0 (socket says 172.21.15.100:53444) 2024-08-17T02:25:14.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4ea4a0700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb4e4142780 0x7fb4e40ae9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:57796/0 (socket says 172.21.15.100:57796) 2024-08-17T02:25:14.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eaca1700 1 -- 172.21.15.100:0/3508093781 learned_addr learned my addr 172.21.15.100:0/3508093781 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:14.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4e9c9f700 1 --2- 172.21.15.100:0/3508093781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e40ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:50094/0 (socket says 172.21.15.100:50094) 2024-08-17T02:25:14.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eaca1700 1 -- 172.21.15.100:0/3508093781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 msgr2=0x7fb4e40ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:14.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eaca1700 1 --2- 172.21.15.100:0/3508093781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e40ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eaca1700 1 -- 172.21.15.100:0/3508093781 >> v1:172.21.15.123:6789/0 conn(0x7fb4e4142780 legacy=0x7fb4e40ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:14.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eaca1700 1 -- 172.21.15.100:0/3508093781 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40c9be0 con 0x7fb4e40cae50 2024-08-17T02:25:14.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4eaca1700 1 --2- 172.21.15.100:0/3508093781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 0x7fb4e415a930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009f20 tx=0x7fb4dc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9af27efd6116d842 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:14.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4e949e700 1 -- 172.21.15.100:0/3508093781 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00e070 con 0x7fb4e40cae50 2024-08-17T02:25:14.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4e949e700 1 -- 172.21.15.100:0/3508093781 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc00b9d0 con 0x7fb4e40cae50 2024-08-17T02:25:14.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.864+0000 7fb4e949e700 1 -- 172.21.15.100:0/3508093781 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc0092e0 con 0x7fb4e40cae50 2024-08-17T02:25:14.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/3508093781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 msgr2=0x7fb4e415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:14.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/3508093781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 0x7fb4e415a930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009f20 tx=0x7fb4dc009840 comp rx=0 tx=0).stop 2024-08-17T02:25:14.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/3508093781 shutdown_connections 2024-08-17T02:25:14.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/3508093781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 0x7fb4e415a930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/3508093781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e40ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/3508093781 >> 172.21.15.100:0/3508093781 conn(0x7fb4e406e070 msgr2=0x7fb4e40b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:14.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/3508093781 shutdown_connections 2024-08-17T02:25:14.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/3508093781 wait complete. 2024-08-17T02:25:14.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 Processor -- start 2024-08-17T02:25:14.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.868+0000 7fb4eb4a2700 1 -- start start 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eb4a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 0x7fb4e4169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eb4a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e4142780 0x7fb4e40b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eb4a2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e416cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eb4a2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40a5780 con 0x7fb4e4142780 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eb4a2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40c65e0 con 0x7fb4e40cae50 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eb4a2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e416d860 con 0x7fb4e4142b50 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e4142780 0x7fb4e40b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e4142780 0x7fb4e40b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50300/0 (socket says 172.21.15.100:50300) 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 -- 172.21.15.100:0/845378590 learned_addr learned my addr 172.21.15.100:0/845378590 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4ea4a0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 0x7fb4e4169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eaca1700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e416cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 -- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 msgr2=0x7fb4e416cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e416cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 -- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 msgr2=0x7fb4e4169650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 0x7fb4e4169650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 -- 172.21.15.100:0/845378590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc009ae0 con 0x7fb4e4142780 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eaca1700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e416cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4e9c9f700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e4142780 0x7fb4e40b0620 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e930 tx=0x7fb4d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4da7fc700 1 -- 172.21.15.100:0/845378590 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4013680 con 0x7fb4e4142780 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4da7fc700 1 -- 172.21.15.100:0/845378590 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d40137e0 con 0x7fb4e4142780 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e40c6880 con 0x7fb4e4142780 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4da7fc700 1 -- 172.21.15.100:0/845378590 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4013b80 con 0x7fb4e4142780 2024-08-17T02:25:14.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.872+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e40c6c10 con 0x7fb4e4142780 2024-08-17T02:25:14.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.876+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e40638f0 con 0x7fb4e4142780 2024-08-17T02:25:14.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.876+0000 7fb4da7fc700 1 -- 172.21.15.100:0/845378590 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb4d400c460 con 0x7fb4e4142780 2024-08-17T02:25:14.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.876+0000 7fb4da7fc700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4c8060860 0x7fb4c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:14.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.876+0000 7fb4da7fc700 1 -- 172.21.15.100:0/845378590 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fb4d407cd60 con 0x7fb4e4142780 2024-08-17T02:25:14.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.876+0000 7fb4ea4a0700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4c8060860 0x7fb4c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:14.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.876+0000 7fb4ea4a0700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4c8060860 0x7fb4c8062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0004fd0 tx=0x7fb4e0007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:14.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:14.884+0000 7fb4da7fc700 1 -- 172.21.15.100:0/845378590 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb4d404d130 con 0x7fb4e4142780 2024-08-17T02:25:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.040+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4e416dba0 con 0x7fb4e4142780 2024-08-17T02:25:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4da7fc700 1 -- 172.21.15.100:0/845378590 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+15562 (secure 0 0 0) 0x7fb4d4050750 con 0x7fb4e4142780 2024-08-17T02:25:15.047 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:15.047 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:25:14.736057+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000351990.meta","create_time":"2024-08-17T02:23:03.999505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000351990"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000351990.data","create_time":"2024-08-17T02:23:04.098167+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000351990"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/3037167008":"2024-08-18T02:25:11.722625+0000","172.21.15.100:6834/3037167008":"2024-08-18T02:25:11.722625+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:25:15.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4c8060860 msgr2=0x7fb4c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4c8060860 0x7fb4c8062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0004fd0 tx=0x7fb4e0007eb0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e4142780 msgr2=0x7fb4e40b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e4142780 0x7fb4e40b0620 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e930 tx=0x7fb4d40091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 shutdown_connections 2024-08-17T02:25:15.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb4c8060860 0x7fb4c8062d10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4e4142b50 0x7fb4e416cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4e4142780 0x7fb4e40b0620 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 --2- 172.21.15.100:0/845378590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e40cae50 0x7fb4e4169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 >> 172.21.15.100:0/845378590 conn(0x7fb4e406e070 msgr2=0x7fb4e413f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:15.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 shutdown_connections 2024-08-17T02:25:15.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.044+0000 7fb4eb4a2700 1 -- 172.21.15.100:0/845378590 wait complete. 2024-08-17T02:25:15.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:21:59.284341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000351990.meta', 'create_time': '2024-08-17T02:23:03.999505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000351990'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000351990.data', 'create_time': '2024-08-17T02:23:04.098167+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000351990'}}}] 2024-08-17T02:25:15.059 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:25:15.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f6d6f700 1 Processor -- start 2024-08-17T02:25:15.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f6d6f700 1 -- start start 2024-08-17T02:25:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f6d6f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 0x7f31f013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f6d6f700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f013be40 con 0x7f31f0155c30 2024-08-17T02:25:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f6d6f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f013d930 con 0x7f31f006a040 2024-08-17T02:25:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f6d6f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f013b230 con 0x7f31f0156000 2024-08-17T02:25:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f5d6d700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f31f0155c30 0x7f31f0069930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58260/0 (socket says 172.21.15.100:58260) 2024-08-17T02:25:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f5d6d700 1 -- 172.21.15.100:0/2943747315 learned_addr learned my addr 172.21.15.100:0/2943747315 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f556c700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 0x7f31f013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449563034 0 0) 0x7f31f013be40 con 0x7f31f0155c30 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d8003570 con 0x7f31f0155c30 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1697655621 0 0) 0x7f31d8003570 con 0x7f31f0155c30 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 msgr2=0x7f31f013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 0x7f31f013a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 >> v1:172.21.15.123:6789/0 conn(0x7f31f006a040 legacy=0x7f31f013aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f013b510 con 0x7f31f0155c30 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f31e80017f0 con 0x7f31f0155c30 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 >> v1:172.21.15.100:6789/0 conn(0x7f31f0155c30 legacy=0x7f31f0069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 0x7f31d8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8003fd0 0x7f31d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8004d50 0x7f31d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f013b230 con 0x7f31d8003fd0 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800bb40 con 0x7f31d8004d50 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800bd00 con 0x7f31f0156000 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f556c700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8003fd0 0x7f31d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f5d6d700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 0x7f31d8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.164+0000 7f31f656e700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8004d50 0x7f31d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f556c700 1 -- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 msgr2=0x7f31d8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f556c700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 0x7f31d8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f556c700 1 -- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8004d50 msgr2=0x7f31d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f556c700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8004d50 0x7f31d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f556c700 1 -- 172.21.15.100:0/2943747315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8003570 con 0x7f31d8003fd0 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f5d6d700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f0156000 0x7f31d8003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f556c700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8003fd0 0x7f31d8004810 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f31f01563f0 tx=0x7f31e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1b8077e8b1eebd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400e710 con 0x7f31d8003fd0 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400e870 con 0x7f31d8003fd0 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f4d6b700 1 -- 172.21.15.100:0/2943747315 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400ec30 con 0x7f31d8003fd0 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f6d6f700 1 -- 172.21.15.100:0/2943747315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8003fd0 msgr2=0x7f31d8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f6d6f700 1 --2- 172.21.15.100:0/2943747315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8003fd0 0x7f31d8004810 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f31f01563f0 tx=0x7f31e40091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f656e700 1 -- 172.21.15.100:0/2943747315 reap_dead start 2024-08-17T02:25:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f6d6f700 1 -- 172.21.15.100:0/2943747315 shutdown_connections 2024-08-17T02:25:15.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f6d6f700 1 -- 172.21.15.100:0/2943747315 >> 172.21.15.100:0/2943747315 conn(0x7f31f006e170 msgr2=0x7f31f006e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:15.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f6d6f700 1 -- 172.21.15.100:0/2943747315 shutdown_connections 2024-08-17T02:25:15.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f6d6f700 1 -- 172.21.15.100:0/2943747315 wait complete. 2024-08-17T02:25:15.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f6d6f700 1 Processor -- start 2024-08-17T02:25:15.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.168+0000 7f31f6d6f700 1 -- start start 2024-08-17T02:25:15.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8003fd0 0x7f31f0166d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f01545c0 0x7f31f0150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31f00a6de0 0x7f31f00a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00a88e0 con 0x7f31f00a6de0 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00a5450 con 0x7f31d8003fd0 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00a77c0 con 0x7f31f01545c0 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31f00a6de0 0x7f31f00a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31f00a6de0 0x7f31f00a7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50312/0 (socket says 172.21.15.100:50312) 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 -- 172.21.15.100:0/262211436 learned_addr learned my addr 172.21.15.100:0/262211436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f5d6d700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8003fd0 0x7f31f0166d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f556c700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f01545c0 0x7f31f0150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 -- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f01545c0 msgr2=0x7f31f0150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f01545c0 0x7f31f0150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 -- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8003fd0 msgr2=0x7f31f0166d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8003fd0 0x7f31f0166d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 -- 172.21.15.100:0/262211436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00a5f60 con 0x7f31f00a6de0 2024-08-17T02:25:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f656e700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31f00a6de0 0x7f31f00a7280 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f31f006a440 tx=0x7f31e0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31edffb700 1 -- 172.21.15.100:0/262211436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0018070 con 0x7f31f00a6de0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31edffb700 1 -- 172.21.15.100:0/262211436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e0006820 con 0x7f31f00a6de0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f0150ac0 con 0x7f31f00a6de0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31edffb700 1 -- 172.21.15.100:0/262211436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0006bc0 con 0x7f31f00a6de0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f00a5910 con 0x7f31f00a6de0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f00a60f0 con 0x7f31f00a6de0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31edffb700 1 -- 172.21.15.100:0/262211436 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f31e0007030 con 0x7f31f00a6de0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.172+0000 7f31edffb700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f31dc060860 0x7f31dc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.176+0000 7f31edffb700 1 -- 172.21.15.100:0/262211436 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f31e007c5e0 con 0x7f31f00a6de0 2024-08-17T02:25:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.176+0000 7f31f5d6d700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f31dc060860 0x7f31dc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.176+0000 7f31f5d6d700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f31dc060860 0x7f31dc062d10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005fd0 tx=0x7f31f013dac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.180+0000 7f31edffb700 1 -- 172.21.15.100:0/262211436 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f31e004c9b0 con 0x7f31f00a6de0 2024-08-17T02:25:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.344+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31f00638f0 con 0x7f31f00a6de0 2024-08-17T02:25:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.344+0000 7f31edffb700 1 -- 172.21.15.100:0/262211436 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31e004ffd0 con 0x7f31f00a6de0 2024-08-17T02:25:15.346 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f31dc060860 msgr2=0x7f31dc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f31dc060860 0x7f31dc062d10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005fd0 tx=0x7f31f013dac0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31f00a6de0 msgr2=0x7f31f00a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31f00a6de0 0x7f31f00a7280 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f31f006a440 tx=0x7f31e0009040 comp rx=0 tx=0).stop 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 shutdown_connections 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f31dc060860 0x7f31dc062d10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31f00a6de0 0x7f31f00a7280 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f01545c0 0x7f31f0150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 --2- 172.21.15.100:0/262211436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31d8003fd0 0x7f31f0166d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 >> 172.21.15.100:0/262211436 conn(0x7f31f006e170 msgr2=0x7f31f00b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:15.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 shutdown_connections 2024-08-17T02:25:15.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.348+0000 7f31f6d6f700 1 -- 172.21.15.100:0/262211436 wait complete. 2024-08-17T02:25:15.364 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000351990.meta pg_num 2024-08-17T02:25:15.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e9417700 1 Processor -- start 2024-08-17T02:25:15.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e9417700 1 -- start start 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e9417700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e4067510 0x7fb6e40ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e9417700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e40ce210 con 0x7fb6e4159480 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e9417700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e40cb440 con 0x7fb6e41590b0 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e9417700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40cba00 con 0x7fb6e4067510 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e8c16700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb6e4159480 0x7fb6e40cebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58276/0 (socket says 172.21.15.100:58276) 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e8c16700 1 -- 172.21.15.100:0/3982374655 learned_addr learned my addr 172.21.15.100:0/3982374655 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e37fe700 1 --2- 172.21.15.100:0/3982374655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e4067510 0x7fb6e40ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e2ffd700 1 -- 172.21.15.100:0/3982374655 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799307938 0 0) 0x7fb6e40ce210 con 0x7fb6e4159480 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e2ffd700 1 -- 172.21.15.100:0/3982374655 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d0003570 con 0x7fb6e4159480 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e2ffd700 1 -- 172.21.15.100:0/3982374655 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520974447 0 0) 0x7fb6e40cb440 con 0x7fb6e41590b0 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e2ffd700 1 -- 172.21.15.100:0/3982374655 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e40ce210 con 0x7fb6e41590b0 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e2ffd700 1 -- 172.21.15.100:0/3982374655 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 149785583 0 0) 0x7fb6d0003570 con 0x7fb6e4159480 2024-08-17T02:25:15.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e37fe700 1 -- 172.21.15.100:0/3982374655 >> v1:172.21.15.123:6789/0 conn(0x7fb6e41590b0 legacy=0x7fb6e4066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e37fe700 1 -- 172.21.15.100:0/3982374655 >> v1:172.21.15.100:6789/0 conn(0x7fb6e4159480 legacy=0x7fb6e40cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.476+0000 7fb6e37fe700 1 -- 172.21.15.100:0/3982374655 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e40cb720 con 0x7fb6e4067510 2024-08-17T02:25:15.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e37fe700 1 --2- 172.21.15.100:0/3982374655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e4067510 0x7fb6e40ce6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009910 tx=0x7fb6cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ef092ae8cc6f3d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e2ffd700 1 -- 172.21.15.100:0/3982374655 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0017f0 con 0x7fb6e4067510 2024-08-17T02:25:15.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e2ffd700 1 -- 172.21.15.100:0/3982374655 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc011420 con 0x7fb6e4067510 2024-08-17T02:25:15.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e2ffd700 1 -- 172.21.15.100:0/3982374655 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0117c0 con 0x7fb6e4067510 2024-08-17T02:25:15.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 -- 172.21.15.100:0/3982374655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e4067510 msgr2=0x7fb6e40ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 --2- 172.21.15.100:0/3982374655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e4067510 0x7fb6e40ce6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009910 tx=0x7fb6cc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 -- 172.21.15.100:0/3982374655 shutdown_connections 2024-08-17T02:25:15.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 --2- 172.21.15.100:0/3982374655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e4067510 0x7fb6e40ce6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 -- 172.21.15.100:0/3982374655 >> 172.21.15.100:0/3982374655 conn(0x7fb6e40d0b90 msgr2=0x7fb6e40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:15.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 -- 172.21.15.100:0/3982374655 shutdown_connections 2024-08-17T02:25:15.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 -- 172.21.15.100:0/3982374655 wait complete. 2024-08-17T02:25:15.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 Processor -- start 2024-08-17T02:25:15.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.480+0000 7fb6e9417700 1 -- start start 2024-08-17T02:25:15.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4067510 0x7fb6e4143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e41590b0 0x7fb6e40b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e4159480 0x7fb6e40b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e415ca60 con 0x7fb6e4067510 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e415bb90 con 0x7fb6e4159480 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e415be50 con 0x7fb6e41590b0 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4067510 0x7fb6e4143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4067510 0x7fb6e4143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50322/0 (socket says 172.21.15.100:50322) 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 -- 172.21.15.100:0/580224318 learned_addr learned my addr 172.21.15.100:0/580224318 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e8c16700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e4159480 0x7fb6e40b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e37fe700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e41590b0 0x7fb6e40b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 -- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e41590b0 msgr2=0x7fb6e40b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e41590b0 0x7fb6e40b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 -- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e4159480 msgr2=0x7fb6e40b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e4159480 0x7fb6e40b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 -- 172.21.15.100:0/580224318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc009420 con 0x7fb6e4067510 2024-08-17T02:25:15.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e3fff700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4067510 0x7fb6e4143940 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400c790 tx=0x7fb6d4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6dbfff700 1 -- 172.21.15.100:0/580224318 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4010510 con 0x7fb6e4067510 2024-08-17T02:25:15.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6dbfff700 1 -- 172.21.15.100:0/580224318 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4010670 con 0x7fb6e4067510 2024-08-17T02:25:15.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e415c130 con 0x7fb6e4067510 2024-08-17T02:25:15.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6dbfff700 1 -- 172.21.15.100:0/580224318 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4010a10 con 0x7fb6e4067510 2024-08-17T02:25:15.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e415c6f0 con 0x7fb6e4067510 2024-08-17T02:25:15.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.484+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e4064330 con 0x7fb6e4067510 2024-08-17T02:25:15.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.488+0000 7fb6dbfff700 1 -- 172.21.15.100:0/580224318 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb6d401b3b0 con 0x7fb6e4067510 2024-08-17T02:25:15.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.488+0000 7fb6dbfff700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb6c4060860 0x7fb6c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.488+0000 7fb6dbfff700 1 -- 172.21.15.100:0/580224318 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fb6d407c600 con 0x7fb6e4067510 2024-08-17T02:25:15.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.488+0000 7fb6e37fe700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb6c4060860 0x7fb6c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.488+0000 7fb6e37fe700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb6c4060860 0x7fb6c4062d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0030c0 tx=0x7fb6cc016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.500+0000 7fb6dbfff700 1 -- 172.21.15.100:0/580224318 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb6d404c930 con 0x7fb6e4067510 2024-08-17T02:25:15.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000351990.meta", "var": "pg_num"} v 0) v1 -- 0x7fb6e4151ba0 con 0x7fb6e4067510 2024-08-17T02:25:15.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6dbfff700 1 -- 172.21.15.100:0/580224318 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000351990.meta", "var": "pg_num"}]=0 v41) v1 ==== 124+0+11 (secure 0 0 0) 0x7fb6d40170c0 con 0x7fb6e4067510 2024-08-17T02:25:15.669 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb6c4060860 msgr2=0x7fb6c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb6c4060860 0x7fb6c4062d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0030c0 tx=0x7fb6cc016000 comp rx=0 tx=0).stop 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4067510 msgr2=0x7fb6e4143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4067510 0x7fb6e4143940 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400c790 tx=0x7fb6d4007e10 comp rx=0 tx=0).stop 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 shutdown_connections 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fb6c4060860 0x7fb6c4062d10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e4159480 0x7fb6e40b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6e41590b0 0x7fb6e40b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 --2- 172.21.15.100:0/580224318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4067510 0x7fb6e4143940 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 >> 172.21.15.100:0/580224318 conn(0x7fb6e40d0b90 msgr2=0x7fb6e40ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:15.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 shutdown_connections 2024-08-17T02:25:15.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.668+0000 7fb6e9417700 1 -- 172.21.15.100:0/580224318 wait complete. 2024-08-17T02:25:15.683 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000351990.data pg_num 2024-08-17T02:25:15.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.788+0000 7fcff1d4e700 1 Processor -- start 2024-08-17T02:25:15.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.788+0000 7fcff1d4e700 1 -- start start 2024-08-17T02:25:15.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfec0aec20 0x7fcfec142a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec0b1180 con 0x7fcfec142660 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec0b0890 con 0x7fcfec0c82e0 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec0c9310 con 0x7fcfec0aec20 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfebfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfec0aec20 0x7fcfec142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfebfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfec0aec20 0x7fcfec142a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50326/0 (socket says 172.21.15.100:50326) 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfebfff700 1 -- 172.21.15.100:0/1270481078 learned_addr learned my addr 172.21.15.100:0/1270481078 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff0d4c700 1 --1- 172.21.15.100:0/1270481078 >> v1:172.21.15.100:6789/0 conn(0x7fcfec142660 0x7fcfec0ae510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58284/0 (socket says 172.21.15.100:58284) 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfebfff700 1 -- 172.21.15.100:0/1270481078 >> v1:172.21.15.123:6789/0 conn(0x7fcfec0c82e0 legacy=0x7fcfec0c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfebfff700 1 -- 172.21.15.100:0/1270481078 >> v1:172.21.15.100:6789/0 conn(0x7fcfec142660 legacy=0x7fcfec0ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfebfff700 1 -- 172.21.15.100:0/1270481078 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0c9030 con 0x7fcfec0aec20 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfebfff700 1 --2- 172.21.15.100:0/1270481078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0aec20 0x7fcfec142a30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40099b0 tx=0x7fcfd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fb45610d2682a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfeb7fe700 1 -- 172.21.15.100:0/1270481078 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd400b860 con 0x7fcfec0aec20 2024-08-17T02:25:15.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfeb7fe700 1 -- 172.21.15.100:0/1270481078 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd400b9c0 con 0x7fcfec0aec20 2024-08-17T02:25:15.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcfeb7fe700 1 -- 172.21.15.100:0/1270481078 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd400bd60 con 0x7fcfec0aec20 2024-08-17T02:25:15.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 -- 172.21.15.100:0/1270481078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0aec20 msgr2=0x7fcfec142a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 --2- 172.21.15.100:0/1270481078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0aec20 0x7fcfec142a30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40099b0 tx=0x7fcfd4009250 comp rx=0 tx=0).stop 2024-08-17T02:25:15.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 -- 172.21.15.100:0/1270481078 shutdown_connections 2024-08-17T02:25:15.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 --2- 172.21.15.100:0/1270481078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0aec20 0x7fcfec142a30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 -- 172.21.15.100:0/1270481078 >> 172.21.15.100:0/1270481078 conn(0x7fcfec0c9830 msgr2=0x7fcfec0c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:15.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 -- 172.21.15.100:0/1270481078 shutdown_connections 2024-08-17T02:25:15.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.792+0000 7fcff1d4e700 1 -- 172.21.15.100:0/1270481078 wait complete. 2024-08-17T02:25:15.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 Processor -- start 2024-08-17T02:25:15.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 -- start start 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfec0aec20 0x7fcfec0c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0c82e0 0x7fcfec0c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfec142660 0x7fcfec131b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec1320d0 con 0x7fcfec0c82e0 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec132f10 con 0x7fcfec0aec20 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec132300 con 0x7fcfec142660 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0c82e0 0x7fcfec0c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0c82e0 0x7fcfec0c3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50332/0 (socket says 172.21.15.100:50332) 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 -- 172.21.15.100:0/4237467564 learned_addr learned my addr 172.21.15.100:0/4237467564 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff0d4c700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfec0aec20 0x7fcfec0c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff154d700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfec142660 0x7fcfec131b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 -- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfec142660 msgr2=0x7fcfec131b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfec142660 0x7fcfec131b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 -- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfec0aec20 msgr2=0x7fcfec0c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfec0aec20 0x7fcfec0c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 -- 172.21.15.100:0/4237467564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd40094f0 con 0x7fcfec0c82e0 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfebfff700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0c82e0 0x7fcfec0c3440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcfec06de30 tx=0x7fcfd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfe8ff9700 1 -- 172.21.15.100:0/4237467564 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4024440 con 0x7fcfec0c82e0 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfe8ff9700 1 -- 172.21.15.100:0/4237467564 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd40245a0 con 0x7fcfec0c82e0 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcfe8ff9700 1 -- 172.21.15.100:0/4237467564 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4024920 con 0x7fcfec0c82e0 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec1325e0 con 0x7fcfec0c82e0 2024-08-17T02:25:15.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.796+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec132ba0 con 0x7fcfec0c82e0 2024-08-17T02:25:15.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.800+0000 7fcfe8ff9700 1 -- 172.21.15.100:0/4237467564 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcfd401a420 con 0x7fcfec0c82e0 2024-08-17T02:25:15.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.800+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec1330a0 con 0x7fcfec0c82e0 2024-08-17T02:25:15.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.800+0000 7fcfe8ff9700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfd0060890 0x7fcfd0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:15.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.800+0000 7fcfe8ff9700 1 -- 172.21.15.100:0/4237467564 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fcfd407ea50 con 0x7fcfec0c82e0 2024-08-17T02:25:15.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.800+0000 7fcff0d4c700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfd0060890 0x7fcfd0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:15.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.800+0000 7fcff0d4c700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfd0060890 0x7fcfd0062d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc004fd0 tx=0x7fcfdc007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:15.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.808+0000 7fcfe8ff9700 1 -- 172.21.15.100:0/4237467564 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcfd404ee20 con 0x7fcfec0c82e0 2024-08-17T02:25:15.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.968+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000351990.data", "var": "pg_num"} v 0) v1 -- 0x7fcfec0638f0 con 0x7fcfec0c82e0 2024-08-17T02:25:15.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.968+0000 7fcfe8ff9700 1 -- 172.21.15.100:0/4237467564 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000351990.data", "var": "pg_num"}]=0 v41) v1 ==== 124+0+11 (secure 0 0 0) 0x7fcfd40150c0 con 0x7fcfec0c82e0 2024-08-17T02:25:15.972 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-08-17T02:25:15.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfd0060890 msgr2=0x7fcfd0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfd0060890 0x7fcfd0062d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc004fd0 tx=0x7fcfdc007eb0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0c82e0 msgr2=0x7fcfec0c3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0c82e0 0x7fcfec0c3440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcfec06de30 tx=0x7fcfd4017000 comp rx=0 tx=0).stop 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 shutdown_connections 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fcfd0060890 0x7fcfd0062d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfec142660 0x7fcfec131b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfec0c82e0 0x7fcfec0c3440 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 --2- 172.21.15.100:0/4237467564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfec0aec20 0x7fcfec0c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 >> 172.21.15.100:0/4237467564 conn(0x7fcfec0c9830 msgr2=0x7fcfec159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 shutdown_connections 2024-08-17T02:25:15.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:15.972+0000 7fcff1d4e700 1 -- 172.21.15.100:0/4237467564 wait complete. 2024-08-17T02:25:15.986 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:25:16.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 Processor -- start 2024-08-17T02:25:16.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 -- start start 2024-08-17T02:25:16.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 0x7f6c1c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6c1c0d0010 0x7f6c1c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6c1c15cea0 0x7f6c1c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c0cf190 con 0x7f6c1c15cea0 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c0d09a0 con 0x7f6c1c0d0010 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0ce5e0 con 0x7f6c1c159040 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6c1c15cea0 0x7f6c1c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6c1c15cea0 0x7f6c1c15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50348/0 (socket says 172.21.15.100:50348) 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c217d2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6c1c0d0010 0x7f6c1c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 -- 172.21.15.100:0/350190572 learned_addr learned my addr 172.21.15.100:0/350190572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c21fd3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 0x7f6c1c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 -- 172.21.15.100:0/350190572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 msgr2=0x7f6c1c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:16.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 --2- 172.21.15.100:0/350190572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 0x7f6c1c0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 -- 172.21.15.100:0/350190572 >> v2:172.21.15.123:3300/0 conn(0x7f6c1c0d0010 msgr2=0x7f6c1c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:16.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 --2- 172.21.15.100:0/350190572 >> v2:172.21.15.123:3300/0 conn(0x7f6c1c0d0010 0x7f6c1c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 -- 172.21.15.100:0/350190572 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0ce860 con 0x7f6c1c15cea0 2024-08-17T02:25:16.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c227d4700 1 --2- 172.21.15.100:0/350190572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c15d320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c067f20 tx=0x7f6c14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917083d19dfcafc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c20fd1700 1 -- 172.21.15.100:0/350190572 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400e070 con 0x7f6c1c15cea0 2024-08-17T02:25:16.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c20fd1700 1 -- 172.21.15.100:0/350190572 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1400b970 con 0x7f6c1c15cea0 2024-08-17T02:25:16.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c20fd1700 1 -- 172.21.15.100:0/350190572 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c140092e0 con 0x7f6c1c15cea0 2024-08-17T02:25:16.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 -- 172.21.15.100:0/350190572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 msgr2=0x7f6c1c15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.088+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/350190572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c15d320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c067f20 tx=0x7f6c14009840 comp rx=0 tx=0).stop 2024-08-17T02:25:16.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 -- 172.21.15.100:0/350190572 shutdown_connections 2024-08-17T02:25:16.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/350190572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c15d320 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/350190572 >> v2:172.21.15.123:3300/0 conn(0x7f6c1c0d0010 0x7f6c1c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/350190572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 0x7f6c1c0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 -- 172.21.15.100:0/350190572 >> 172.21.15.100:0/350190572 conn(0x7f6c1c068250 msgr2=0x7f6c1c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:16.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 -- 172.21.15.100:0/350190572 shutdown_connections 2024-08-17T02:25:16.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 -- 172.21.15.100:0/350190572 wait complete. 2024-08-17T02:25:16.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 Processor -- start 2024-08-17T02:25:16.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 -- start start 2024-08-17T02:25:16.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c1c0d0010 0x7f6c1c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 0x7f6c1c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c063270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1695c0 con 0x7f6c1c15cea0 2024-08-17T02:25:16.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0b0390 con 0x7f6c1c0d0010 2024-08-17T02:25:16.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c22fd5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0b0670 con 0x7f6c1c159040 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c227d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c063270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c227d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c063270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50350/0 (socket says 172.21.15.100:50350) 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c227d4700 1 -- 172.21.15.100:0/494173916 learned_addr learned my addr 172.21.15.100:0/494173916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c217d2700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 0x7f6c1c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.092+0000 7f6c21fd3700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c1c0d0010 0x7f6c1c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c227d4700 1 -- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 msgr2=0x7f6c1c152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c227d4700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 0x7f6c1c152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c227d4700 1 -- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c1c0d0010 msgr2=0x7f6c1c0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c227d4700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c1c0d0010 0x7f6c1c0afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c227d4700 1 -- 172.21.15.100:0/494173916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c14009ae0 con 0x7f6c1c15cea0 2024-08-17T02:25:16.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c227d4700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c063270 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c088ef0 tx=0x7f6c14018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c11ffb700 1 -- 172.21.15.100:0/494173916 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14008c30 con 0x7f6c1c15cea0 2024-08-17T02:25:16.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c11ffb700 1 -- 172.21.15.100:0/494173916 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c14008d90 con 0x7f6c1c15cea0 2024-08-17T02:25:16.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c0b0950 con 0x7f6c1c15cea0 2024-08-17T02:25:16.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c11ffb700 1 -- 172.21.15.100:0/494173916 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c140266b0 con 0x7f6c1c15cea0 2024-08-17T02:25:16.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c0aff60 con 0x7f6c1c15cea0 2024-08-17T02:25:16.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c169770 con 0x7f6c1c15cea0 2024-08-17T02:25:16.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.096+0000 7f6c11ffb700 1 -- 172.21.15.100:0/494173916 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6c14003040 con 0x7f6c1c15cea0 2024-08-17T02:25:16.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.100+0000 7f6c11ffb700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c00060860 0x7f6c00062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.100+0000 7f6c11ffb700 1 -- 172.21.15.100:0/494173916 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f6c1407ea20 con 0x7f6c1c15cea0 2024-08-17T02:25:16.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.100+0000 7f6c21fd3700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c00060860 0x7f6c00062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.100+0000 7f6c21fd3700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c00060860 0x7f6c00062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009d40 tx=0x7f6c0c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.108+0000 7f6c11ffb700 1 -- 172.21.15.100:0/494173916 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6c1c169770 con 0x7f6c1c15cea0 2024-08-17T02:25:16.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.308+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c1c0643a0 con 0x7f6c1c15cea0 2024-08-17T02:25:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.308+0000 7f6c11ffb700 1 -- 172.21.15.100:0/494173916 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4627 (secure 0 0 0) 0x7f6c1404edf0 con 0x7f6c1c15cea0 2024-08-17T02:25:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:16.312 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1877360383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1877360383},{"type":"v1","addr":"172.21.15.100:6835","nonce":1877360383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:23:05.071644+0000","modified":"2024-08-17T02:23:06.938110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000351990","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:25:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c00060860 msgr2=0x7f6c00062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c00060860 0x7f6c00062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009d40 tx=0x7f6c0c008040 comp rx=0 tx=0).stop 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 msgr2=0x7f6c1c063270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c063270 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c088ef0 tx=0x7f6c14018000 comp rx=0 tx=0).stop 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 shutdown_connections 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c00060860 0x7f6c00062d10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c1c15cea0 0x7f6c1c063270 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c1c159040 0x7f6c1c152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 --2- 172.21.15.100:0/494173916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c1c0d0010 0x7f6c1c0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 >> 172.21.15.100:0/494173916 conn(0x7f6c1c068250 msgr2=0x7f6c1c0aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 shutdown_connections 2024-08-17T02:25:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.312+0000 7f6c22fd5700 1 -- 172.21.15.100:0/494173916 wait complete. 2024-08-17T02:25:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-08-17T02:25:16.327 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:25:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.428+0000 7fe178ff6700 1 Processor -- start 2024-08-17T02:25:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.428+0000 7fe178ff6700 1 -- start start 2024-08-17T02:25:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.428+0000 7fe178ff6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe174156f80 0x7fe1740ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.428+0000 7fe178ff6700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1740ace80 0x7fe174157350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.428+0000 7fe178ff6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1740adf50 con 0x7fe174157890 2024-08-17T02:25:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.428+0000 7fe178ff6700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740ae920 con 0x7fe1740ace80 2024-08-17T02:25:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.428+0000 7fe178ff6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740ad340 con 0x7fe174156f80 2024-08-17T02:25:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe16bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1740ace80 0x7fe174157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe16bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1740ace80 0x7fe174157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50362/0 (socket says 172.21.15.100:50362) 2024-08-17T02:25:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe17259c700 1 --2- 172.21.15.100:0/2527597419 >> v2:172.21.15.123:3300/0 conn(0x7fe174156f80 0x7fe1740ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe16bfff700 1 -- 172.21.15.100:0/2527597419 learned_addr learned my addr 172.21.15.100:0/2527597419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe171d9b700 1 -- 172.21.15.100:0/2527597419 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599855314 0 0) 0x7fe1740adf50 con 0x7fe174157890 2024-08-17T02:25:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe171d9b700 1 -- 172.21.15.100:0/2527597419 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe160003570 con 0x7fe174157890 2024-08-17T02:25:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe16bfff700 1 -- 172.21.15.100:0/2527597419 >> v2:172.21.15.123:3300/0 conn(0x7fe174156f80 msgr2=0x7fe1740ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe16bfff700 1 --2- 172.21.15.100:0/2527597419 >> v2:172.21.15.123:3300/0 conn(0x7fe174156f80 0x7fe1740ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe16bfff700 1 -- 172.21.15.100:0/2527597419 >> v1:172.21.15.123:6789/0 conn(0x7fe174157890 legacy=0x7fe1740aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe16bfff700 1 -- 172.21.15.100:0/2527597419 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740ad620 con 0x7fe1740ace80 2024-08-17T02:25:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe17259c700 1 --2- 172.21.15.100:0/2527597419 >> v2:172.21.15.123:3300/0 conn(0x7fe174156f80 0x7fe1740ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe16bfff700 1 --2- 172.21.15.100:0/2527597419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174157350 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe17406c3b0 tx=0x7fe15c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3181ee39ccb23c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe171d9b700 1 -- 172.21.15.100:0/2527597419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00b640 con 0x7fe1740ace80 2024-08-17T02:25:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe171d9b700 1 -- 172.21.15.100:0/2527597419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c00b7a0 con 0x7fe1740ace80 2024-08-17T02:25:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe171d9b700 1 -- 172.21.15.100:0/2527597419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00bb60 con 0x7fe1740ace80 2024-08-17T02:25:16.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe178ff6700 1 -- 172.21.15.100:0/2527597419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 msgr2=0x7fe174157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2527597419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174157350 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe17406c3b0 tx=0x7fe15c009250 comp rx=0 tx=0).stop 2024-08-17T02:25:16.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe178ff6700 1 -- 172.21.15.100:0/2527597419 shutdown_connections 2024-08-17T02:25:16.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2527597419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174157350 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2527597419 >> v2:172.21.15.123:3300/0 conn(0x7fe174156f80 0x7fe1740ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe178ff6700 1 -- 172.21.15.100:0/2527597419 >> 172.21.15.100:0/2527597419 conn(0x7fe174066e00 msgr2=0x7fe174067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe178ff6700 1 -- 172.21.15.100:0/2527597419 shutdown_connections 2024-08-17T02:25:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.432+0000 7fe178ff6700 1 -- 172.21.15.100:0/2527597419 wait complete. 2024-08-17T02:25:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 Processor -- start 2024-08-17T02:25:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 -- start start 2024-08-17T02:25:16.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174140690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe174156f80 0x7fe17414e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe174157890 0x7fe17414ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17414f440 con 0x7fe1740ace80 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17414ffc0 con 0x7fe174156f80 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17414f690 con 0x7fe174157890 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174140690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50366/0 (socket says 172.21.15.100:50366) 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 -- 172.21.15.100:0/2177882870 learned_addr learned my addr 172.21.15.100:0/2177882870 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe16bfff700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe174156f80 0x7fe17414e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe172d9d700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe174157890 0x7fe17414ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 -- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe174157890 msgr2=0x7fe17414ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe174157890 0x7fe17414ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 -- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe174156f80 msgr2=0x7fe17414e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe174156f80 0x7fe17414e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 -- 172.21.15.100:0/2177882870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe164009420 con 0x7fe1740ace80 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe172d9d700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe174157890 0x7fe17414ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe17259c700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174140690 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe16400ecb0 tx=0x7fe1640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe16a7fc700 1 -- 172.21.15.100:0/2177882870 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400c950 con 0x7fe1740ace80 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe16a7fc700 1 -- 172.21.15.100:0/2177882870 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16400cab0 con 0x7fe1740ace80 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c0094f0 con 0x7fe1740ace80 2024-08-17T02:25:16.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe16a7fc700 1 -- 172.21.15.100:0/2177882870 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400ce30 con 0x7fe1740ace80 2024-08-17T02:25:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.436+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17414fc50 con 0x7fe1740ace80 2024-08-17T02:25:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.440+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1740645c0 con 0x7fe1740ace80 2024-08-17T02:25:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.440+0000 7fe16a7fc700 1 -- 172.21.15.100:0/2177882870 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe164013680 con 0x7fe1740ace80 2024-08-17T02:25:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.440+0000 7fe16a7fc700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe154060860 0x7fe154062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.440+0000 7fe16a7fc700 1 -- 172.21.15.100:0/2177882870 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fe16407d000 con 0x7fe1740ace80 2024-08-17T02:25:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.440+0000 7fe16bfff700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe154060860 0x7fe154062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.440+0000 7fe16bfff700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe154060860 0x7fe154062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe17406c3b0 tx=0x7fe15c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.444+0000 7fe16a7fc700 1 -- 172.21.15.100:0/2177882870 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe16404d3d0 con 0x7fe1740ace80 2024-08-17T02:25:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.604+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe17406cb00 con 0x7fe1740ace80 2024-08-17T02:25:16.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe16a7fc700 1 -- 172.21.15.100:0/2177882870 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+15562 (secure 0 0 0) 0x7fe1640509f0 con 0x7fe1740ace80 2024-08-17T02:25:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:16.610 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:25:14.736057+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000351990.meta","create_time":"2024-08-17T02:23:03.999505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000351990"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000351990.data","create_time":"2024-08-17T02:23:04.098167+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000351990"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/3037167008":"2024-08-18T02:25:11.722625+0000","172.21.15.100:6834/3037167008":"2024-08-18T02:25:11.722625+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:25:16.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe154060860 msgr2=0x7fe154062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe154060860 0x7fe154062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe17406c3b0 tx=0x7fe15c017000 comp rx=0 tx=0).stop 2024-08-17T02:25:16.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 msgr2=0x7fe174140690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174140690 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe16400ecb0 tx=0x7fe1640091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 shutdown_connections 2024-08-17T02:25:16.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe154060860 0x7fe154062d10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe174157890 0x7fe17414ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe174156f80 0x7fe17414e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 --2- 172.21.15.100:0/2177882870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1740ace80 0x7fe174140690 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 >> 172.21.15.100:0/2177882870 conn(0x7fe174066e00 msgr2=0x7fe17406afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:16.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 shutdown_connections 2024-08-17T02:25:16.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.608+0000 7fe178ff6700 1 -- 172.21.15.100:0/2177882870 wait complete. 2024-08-17T02:25:16.625 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000351990 and related pools 2024-08-17T02:25:16.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:25:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.732+0000 7f98fb59e700 1 Processor -- start 2024-08-17T02:25:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.732+0000 7f98fb59e700 1 -- start start 2024-08-17T02:25:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.732+0000 7f98fb59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98fc068c10 0x7f98fc158a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.732+0000 7f98fb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc0ab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.732+0000 7f98fb59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc0aa270 con 0x7f98fc068840 2024-08-17T02:25:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.732+0000 7f98fb59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc069090 con 0x7f98fc068c10 2024-08-17T02:25:16.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.732+0000 7f98fb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0aa550 con 0x7f98fc158fa0 2024-08-17T02:25:16.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f9d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98fc068c10 0x7f98fc158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f9d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98fc068c10 0x7f98fc158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:53516/0 (socket says 172.21.15.100:53516) 2024-08-17T02:25:16.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f9d9b700 1 -- 172.21.15.100:0/1809378413 learned_addr learned my addr 172.21.15.100:0/1809378413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:16.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fad9d700 1 --2- 172.21.15.100:0/1809378413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc0ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f9d9b700 1 -- 172.21.15.100:0/1809378413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 msgr2=0x7f98fc0ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f9d9b700 1 --2- 172.21.15.100:0/1809378413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc0ab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f9d9b700 1 -- 172.21.15.100:0/1809378413 >> v1:172.21.15.123:6789/0 conn(0x7f98fc068840 legacy=0x7f98fc0a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f9d9b700 1 -- 172.21.15.100:0/1809378413 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0aa830 con 0x7f98fc068c10 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f9d9b700 1 --2- 172.21.15.100:0/1809378413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068c10 0x7f98fc158a60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009f50 tx=0x7f98e4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f79ec3c7d22a785 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f959a700 1 -- 172.21.15.100:0/1809378413 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400e070 con 0x7f98fc068c10 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f959a700 1 -- 172.21.15.100:0/1809378413 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e400b9c0 con 0x7f98fc068c10 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98f959a700 1 -- 172.21.15.100:0/1809378413 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40092e0 con 0x7f98fc068c10 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 -- 172.21.15.100:0/1809378413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068c10 msgr2=0x7f98fc158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 --2- 172.21.15.100:0/1809378413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068c10 0x7f98fc158a60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009f50 tx=0x7f98e4009840 comp rx=0 tx=0).stop 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 -- 172.21.15.100:0/1809378413 shutdown_connections 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 --2- 172.21.15.100:0/1809378413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc0ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 --2- 172.21.15.100:0/1809378413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068c10 0x7f98fc158a60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 -- 172.21.15.100:0/1809378413 >> 172.21.15.100:0/1809378413 conn(0x7f98fc06d800 msgr2=0x7f98fc06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:16.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 -- 172.21.15.100:0/1809378413 shutdown_connections 2024-08-17T02:25:16.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 -- 172.21.15.100:0/1809378413 wait complete. 2024-08-17T02:25:16.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 Processor -- start 2024-08-17T02:25:16.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.736+0000 7f98fb59e700 1 -- start start 2024-08-17T02:25:16.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068840 0x7f98fc15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fb59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc068c10 0x7f98fc0c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fb59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15b780 con 0x7f98fc068c10 2024-08-17T02:25:16.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15ab70 con 0x7f98fc068840 2024-08-17T02:25:16.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15ae50 con 0x7f98fc158fa0 2024-08-17T02:25:16.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98f9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc068c10 0x7f98fc0c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98f9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc068c10 0x7f98fc0c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50368/0 (socket says 172.21.15.100:50368) 2024-08-17T02:25:16.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98f9d9b700 1 -- 172.21.15.100:0/2564033134 learned_addr learned my addr 172.21.15.100:0/2564033134 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:16.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fad9d700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fa59c700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068840 0x7f98fc15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fad9d700 1 -- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068840 msgr2=0x7f98fc15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fad9d700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068840 0x7f98fc15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fad9d700 1 -- 172.21.15.100:0/2564033134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc068c10 msgr2=0x7f98fc0c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fad9d700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc068c10 0x7f98fc0c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fad9d700 1 -- 172.21.15.100:0/2564033134 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e4009ae0 con 0x7f98fc158fa0 2024-08-17T02:25:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98f9d9b700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc068c10 0x7f98fc0c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fad9d700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc157150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e900 tx=0x7f98f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98f27fc700 1 -- 172.21.15.100:0/2564033134 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4013680 con 0x7f98fc158fa0 2024-08-17T02:25:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98f27fc700 1 -- 172.21.15.100:0/2564033134 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4013820 con 0x7f98fc158fa0 2024-08-17T02:25:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98f27fc700 1 -- 172.21.15.100:0/2564033134 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4013be0 con 0x7f98fc158fa0 2024-08-17T02:25:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc15b130 con 0x7f98fc158fa0 2024-08-17T02:25:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.740+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc15c150 con 0x7f98fc158fa0 2024-08-17T02:25:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.744+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc15b8f0 con 0x7f98fc158fa0 2024-08-17T02:25:16.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.744+0000 7f98f27fc700 1 -- 172.21.15.100:0/2564033134 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f98f40123d0 con 0x7f98fc158fa0 2024-08-17T02:25:16.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.744+0000 7f98f27fc700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f98e0060860 0x7f98e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:16.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.744+0000 7f98f27fc700 1 -- 172.21.15.100:0/2564033134 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f98f407c930 con 0x7f98fc158fa0 2024-08-17T02:25:16.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.744+0000 7f98fa59c700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f98e0060860 0x7f98e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:16.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.744+0000 7f98fa59c700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f98e0060860 0x7f98e0062d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98ec003aa0 tx=0x7f98ec000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.752+0000 7f98f27fc700 1 -- 172.21.15.100:0/2564033134 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f98f4050790 con 0x7f98fc158fa0 2024-08-17T02:25:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.956+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98fc15af90 con 0x7f98fc158fa0 2024-08-17T02:25:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.956+0000 7f98f27fc700 1 -- 172.21.15.100:0/2564033134 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4627 (secure 0 0 0) 0x7f98fc15af90 con 0x7f98fc158fa0 2024-08-17T02:25:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:16.960 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1877360383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1877360383},{"type":"v1","addr":"172.21.15.100:6835","nonce":1877360383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:23:05.071644+0000","modified":"2024-08-17T02:23:06.938110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000351990","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:25:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f98e0060860 msgr2=0x7f98e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f98e0060860 0x7f98e0062d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98ec003aa0 tx=0x7f98ec000bc0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 msgr2=0x7f98fc157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc157150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e900 tx=0x7f98f40091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 shutdown_connections 2024-08-17T02:25:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f98e0060860 0x7f98e0062d10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98fc158fa0 0x7f98fc157150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc068c10 0x7f98fc0c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 --2- 172.21.15.100:0/2564033134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98fc068840 0x7f98fc15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 >> 172.21.15.100:0/2564033134 conn(0x7f98fc06d800 msgr2=0x7f98fc0ac0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 shutdown_connections 2024-08-17T02:25:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:16.960+0000 7f98fb59e700 1 -- 172.21.15.100:0/2564033134 wait complete. 2024-08-17T02:25:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-08-17T02:25:16.974 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:25:17.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 Processor -- start 2024-08-17T02:25:17.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- start start 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66b40af030 0x7f66b40a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f66b40a9260 0x7f66b40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b40b0270 con 0x7f66b415d540 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b40af660 con 0x7f66b40af030 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b40af940 con 0x7f66b40a9260 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b259c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66b40af030 0x7f66b40a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b259c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66b40af030 0x7f66b40a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50382/0 (socket says 172.21.15.100:50382) 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b259c700 1 -- 172.21.15.100:0/892196015 learned_addr learned my addr 172.21.15.100:0/892196015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b359e700 1 --2- 172.21.15.100:0/892196015 >> v2:172.21.15.123:3300/0 conn(0x7f66b40a9260 0x7f66b40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b259c700 1 -- 172.21.15.100:0/892196015 >> v2:172.21.15.123:3300/0 conn(0x7f66b40a9260 msgr2=0x7f66b40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b259c700 1 --2- 172.21.15.100:0/892196015 >> v2:172.21.15.123:3300/0 conn(0x7f66b40a9260 0x7f66b40b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b259c700 1 -- 172.21.15.100:0/892196015 >> v1:172.21.15.100:6789/0 conn(0x7f66b415d540 legacy=0x7f66b406e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b259c700 1 -- 172.21.15.100:0/892196015 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40afc20 con 0x7f66b40af030 2024-08-17T02:25:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b259c700 1 --2- 172.21.15.100:0/892196015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b40a8d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f66b406ccc0 tx=0x7f66ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9b3373ce320013 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b1d9b700 1 -- 172.21.15.100:0/892196015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00e070 con 0x7f66b40af030 2024-08-17T02:25:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b1d9b700 1 -- 172.21.15.100:0/892196015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac00ba50 con 0x7f66b40af030 2024-08-17T02:25:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b1d9b700 1 -- 172.21.15.100:0/892196015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0092e0 con 0x7f66b40af030 2024-08-17T02:25:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- 172.21.15.100:0/892196015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 msgr2=0x7f66b40a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 --2- 172.21.15.100:0/892196015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b40a8d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f66b406ccc0 tx=0x7f66ac009840 comp rx=0 tx=0).stop 2024-08-17T02:25:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- 172.21.15.100:0/892196015 shutdown_connections 2024-08-17T02:25:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 --2- 172.21.15.100:0/892196015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b40a8d20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 --2- 172.21.15.100:0/892196015 >> v2:172.21.15.123:3300/0 conn(0x7f66b40a9260 0x7f66b40b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- 172.21.15.100:0/892196015 >> 172.21.15.100:0/892196015 conn(0x7f66b4066e00 msgr2=0x7f66b4067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- 172.21.15.100:0/892196015 shutdown_connections 2024-08-17T02:25:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.076+0000 7f66b9242700 1 -- 172.21.15.100:0/892196015 wait complete. 2024-08-17T02:25:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 Processor -- start 2024-08-17T02:25:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 -- start start 2024-08-17T02:25:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66b40a9260 0x7f66b4099500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b4161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b415d540 0x7f66b4158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b4161f30 con 0x7f66b40af030 2024-08-17T02:25:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40c93b0 con 0x7f66b415d540 2024-08-17T02:25:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b4161c90 con 0x7f66b40a9260 2024-08-17T02:25:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b4161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b4161750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50394/0 (socket says 172.21.15.100:50394) 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 -- 172.21.15.100:0/3858437964 learned_addr learned my addr 172.21.15.100:0/3858437964 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b359e700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b415d540 0x7f66b4158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b2d9d700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66b40a9260 0x7f66b4099500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 -- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66b40a9260 msgr2=0x7f66b4099500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66b40a9260 0x7f66b4099500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 -- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b415d540 msgr2=0x7f66b4158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b415d540 0x7f66b4158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 -- 172.21.15.100:0/3858437964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac009ae0 con 0x7f66b40af030 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b259c700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b4161750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f66b406f870 tx=0x7f66ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f669affd700 1 -- 172.21.15.100:0/3858437964 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac008a00 con 0x7f66b40af030 2024-08-17T02:25:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f669affd700 1 -- 172.21.15.100:0/3858437964 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac008b60 con 0x7f66b40af030 2024-08-17T02:25:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b40c8a80 con 0x7f66b40af030 2024-08-17T02:25:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b40c9040 con 0x7f66b40af030 2024-08-17T02:25:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.080+0000 7f669affd700 1 -- 172.21.15.100:0/3858437964 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac026460 con 0x7f66b40af030 2024-08-17T02:25:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.084+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b40969f0 con 0x7f66b40af030 2024-08-17T02:25:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.084+0000 7f669affd700 1 -- 172.21.15.100:0/3858437964 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f66ac003040 con 0x7f66b40af030 2024-08-17T02:25:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.084+0000 7f669affd700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66a0060860 0x7f66a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.084+0000 7f669affd700 1 -- 172.21.15.100:0/3858437964 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f66ac07e9e0 con 0x7f66b40af030 2024-08-17T02:25:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.084+0000 7f66b2d9d700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66a0060860 0x7f66a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.084+0000 7f66b2d9d700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66a0060860 0x7f66a0062d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f66a8003950 tx=0x7f66a8007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.092+0000 7f669affd700 1 -- 172.21.15.100:0/3858437964 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f66b40969f0 con 0x7f66b40af030 2024-08-17T02:25:17.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.288+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66b40969f0 con 0x7f66b40af030 2024-08-17T02:25:17.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.288+0000 7f669affd700 1 -- 172.21.15.100:0/3858437964 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4627 (secure 0 0 0) 0x7f66b40969f0 con 0x7f66b40af030 2024-08-17T02:25:17.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:17.291 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1877360383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1877360383},{"type":"v1","addr":"172.21.15.100:6835","nonce":1877360383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:23:05.071644+0000","modified":"2024-08-17T02:23:06.938110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6833/1584700240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1584700240},{"type":"v1","addr":"172.21.15.100:6833","nonce":1584700240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000351990","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:25:17.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66a0060860 msgr2=0x7f66a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66a0060860 0x7f66a0062d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f66a8003950 tx=0x7f66a8007ec0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 msgr2=0x7f66b4161750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b4161750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f66b406f870 tx=0x7f66ac018000 comp rx=0 tx=0).stop 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 shutdown_connections 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f66a0060860 0x7f66a0062d10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b415d540 0x7f66b4158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40af030 0x7f66b4161750 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 --2- 172.21.15.100:0/3858437964 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66b40a9260 0x7f66b4099500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 >> 172.21.15.100:0/3858437964 conn(0x7f66b4066e00 msgr2=0x7f66b406a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 shutdown_connections 2024-08-17T02:25:17.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.292+0000 7f66b9242700 1 -- 172.21.15.100:0/3858437964 wait complete. 2024-08-17T02:25:17.295 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-08-17T02:25:17.306 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:25:17.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.404+0000 7f967060c700 1 Processor -- start 2024-08-17T02:25:17.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.404+0000 7f967060c700 1 -- start start 2024-08-17T02:25:17.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f967060c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96680c9820 0x7f9668159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f967060c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f966815a090 0x7f96680ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f967060c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96680c9e00 con 0x7f9668153a40 2024-08-17T02:25:17.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f967060c700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96680ccd20 con 0x7f96680c9820 2024-08-17T02:25:17.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f967060c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96680cc890 con 0x7f966815a090 2024-08-17T02:25:17.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966dba7700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96680c9820 0x7f9668159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966dba7700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96680c9820 0x7f9668159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50400/0 (socket says 172.21.15.100:50400) 2024-08-17T02:25:17.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966dba7700 1 -- 172.21.15.100:0/2928574053 learned_addr learned my addr 172.21.15.100:0/2928574053 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:17.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966eba9700 1 --2- 172.21.15.100:0/2928574053 >> v2:172.21.15.123:3300/0 conn(0x7f966815a090 0x7f96680ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966dba7700 1 -- 172.21.15.100:0/2928574053 >> v2:172.21.15.123:3300/0 conn(0x7f966815a090 msgr2=0x7f96680ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:17.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966dba7700 1 --2- 172.21.15.100:0/2928574053 >> v2:172.21.15.123:3300/0 conn(0x7f966815a090 0x7f96680ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966dba7700 1 -- 172.21.15.100:0/2928574053 >> v1:172.21.15.123:6789/0 conn(0x7f9668153a40 legacy=0x7f96680b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966dba7700 1 -- 172.21.15.100:0/2928574053 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966815a480 con 0x7f96680c9820 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966dba7700 1 --2- 172.21.15.100:0/2928574053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96680c9820 0x7f9668159b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96680b2960 tx=0x7f9658009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b5f1c57951b079 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966d3a6700 1 -- 172.21.15.100:0/2928574053 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658009e70 con 0x7f96680c9820 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966d3a6700 1 -- 172.21.15.100:0/2928574053 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965800b860 con 0x7f96680c9820 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.408+0000 7f966d3a6700 1 -- 172.21.15.100:0/2928574053 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800bc00 con 0x7f96680c9820 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- 172.21.15.100:0/2928574053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96680c9820 msgr2=0x7f9668159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 --2- 172.21.15.100:0/2928574053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96680c9820 0x7f9668159b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96680b2960 tx=0x7f9658009250 comp rx=0 tx=0).stop 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- 172.21.15.100:0/2928574053 shutdown_connections 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 --2- 172.21.15.100:0/2928574053 >> v2:172.21.15.123:3300/0 conn(0x7f966815a090 0x7f96680ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 --2- 172.21.15.100:0/2928574053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96680c9820 0x7f9668159b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- 172.21.15.100:0/2928574053 >> 172.21.15.100:0/2928574053 conn(0x7f96680a9db0 msgr2=0x7f96680ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:17.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- 172.21.15.100:0/2928574053 shutdown_connections 2024-08-17T02:25:17.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- 172.21.15.100:0/2928574053 wait complete. 2024-08-17T02:25:17.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 Processor -- start 2024-08-17T02:25:17.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- start start 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96680c9820 0x7f9668150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9668153a40 0x7f9668067690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f966815a090 0x7f9668067bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9668099520 con 0x7f966815a090 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9668099800 con 0x7f96680c9820 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.412+0000 7f967060c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9668099ae0 con 0x7f9668153a40 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f966815a090 0x7f9668067bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f966815a090 0x7f9668067bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50404/0 (socket says 172.21.15.100:50404) 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966dba7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9668153a40 0x7f9668067690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966e3a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96680c9820 0x7f9668150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 -- 172.21.15.100:0/3387485321 learned_addr learned my addr 172.21.15.100:0/3387485321 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 -- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9668153a40 msgr2=0x7f9668067690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9668153a40 0x7f9668067690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 -- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96680c9820 msgr2=0x7f9668150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96680c9820 0x7f9668150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 -- 172.21.15.100:0/3387485321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580094f0 con 0x7f966815a090 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966e3a8700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96680c9820 0x7f9668150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f966eba9700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f966815a090 0x7f9668067bd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9660009e90 tx=0x7f966000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f965e7fc700 1 -- 172.21.15.100:0/3387485321 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966000cb70 con 0x7f966815a090 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f965e7fc700 1 -- 172.21.15.100:0/3387485321 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966000f040 con 0x7f966815a090 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9668099dc0 con 0x7f966815a090 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f965e7fc700 1 -- 172.21.15.100:0/3387485321 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660014480 con 0x7f966815a090 2024-08-17T02:25:17.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.416+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9668151180 con 0x7f966815a090 2024-08-17T02:25:17.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.420+0000 7f965e7fc700 1 -- 172.21.15.100:0/3387485321 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9660014830 con 0x7f966815a090 2024-08-17T02:25:17.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.420+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966809a2a0 con 0x7f966815a090 2024-08-17T02:25:17.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.420+0000 7f965e7fc700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f964c060860 0x7f964c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.420+0000 7f965e7fc700 1 -- 172.21.15.100:0/3387485321 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f966007d650 con 0x7f966815a090 2024-08-17T02:25:17.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.420+0000 7f966e3a8700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f964c060860 0x7f964c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.420+0000 7f966e3a8700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f964c060860 0x7f964c062d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9664001d20 tx=0x7f9664009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.428+0000 7f965e7fc700 1 -- 172.21.15.100:0/3387485321 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f966004da20 con 0x7f966815a090 2024-08-17T02:25:17.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.592+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96680645c0 con 0x7f966815a090 2024-08-17T02:25:17.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.592+0000 7f965e7fc700 1 -- 172.21.15.100:0/3387485321 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+15562 (secure 0 0 0) 0x7f9660051040 con 0x7f966815a090 2024-08-17T02:25:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:17.596 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:25:14.736057+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000351990.meta","create_time":"2024-08-17T02:23:03.999505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000351990"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000351990.data","create_time":"2024-08-17T02:23:04.098167+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000351990"}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/3037167008":"2024-08-18T02:25:11.722625+0000","172.21.15.100:6834/3037167008":"2024-08-18T02:25:11.722625+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:25:17.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f964c060860 msgr2=0x7f964c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f964c060860 0x7f964c062d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9664001d20 tx=0x7f9664009040 comp rx=0 tx=0).stop 2024-08-17T02:25:17.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f966815a090 msgr2=0x7f9668067bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f966815a090 0x7f9668067bd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9660009e90 tx=0x7f966000b040 comp rx=0 tx=0).stop 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 shutdown_connections 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f964c060860 0x7f964c062d10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f966815a090 0x7f9668067bd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9668153a40 0x7f9668067690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 --2- 172.21.15.100:0/3387485321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96680c9820 0x7f9668150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 >> 172.21.15.100:0/3387485321 conn(0x7f96680a9db0 msgr2=0x7f9668151a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 shutdown_connections 2024-08-17T02:25:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.596+0000 7f967060c700 1 -- 172.21.15.100:0/3387485321 wait complete. 2024-08-17T02:25:17.611 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000351990 2024-08-17T02:25:17.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.712+0000 7f3af6c7b700 1 Processor -- start 2024-08-17T02:25:17.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.712+0000 7f3af6c7b700 1 -- start start 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3af0155140 0x7f3af0155510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3af006e9c0 0x7f3af00aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af00aa890 con 0x7f3af00aed20 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af00a9fd0 con 0x7f3af006e9c0 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af0159cb0 con 0x7f3af0155140 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af5218700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3af006e9c0 0x7f3af00aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3ae7fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3af00aed20 0x7f3af006e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58294/0 (socket says 172.21.15.100:58294) 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3ae7fff700 1 -- 172.21.15.100:0/400882386 learned_addr learned my addr 172.21.15.100:0/400882386 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:17.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af4a17700 1 --2- 172.21.15.100:0/400882386 >> v2:172.21.15.123:3300/0 conn(0x7f3af0155140 0x7f3af0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3ae77fe700 1 -- 172.21.15.100:0/400882386 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840737383 0 0) 0x7f3af00aa890 con 0x7f3af00aed20 2024-08-17T02:25:17.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3ae77fe700 1 -- 172.21.15.100:0/400882386 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ad8003570 con 0x7f3af00aed20 2024-08-17T02:25:17.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af5218700 1 -- 172.21.15.100:0/400882386 >> v2:172.21.15.123:3300/0 conn(0x7f3af0155140 msgr2=0x7f3af0155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af5218700 1 --2- 172.21.15.100:0/400882386 >> v2:172.21.15.123:3300/0 conn(0x7f3af0155140 0x7f3af0155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af5218700 1 -- 172.21.15.100:0/400882386 >> v1:172.21.15.100:6789/0 conn(0x7f3af00aed20 legacy=0x7f3af006e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af5218700 1 -- 172.21.15.100:0/400882386 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af00aa240 con 0x7f3af006e9c0 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af4a17700 1 --2- 172.21.15.100:0/400882386 >> v2:172.21.15.123:3300/0 conn(0x7f3af0155140 0x7f3af0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af5218700 1 --2- 172.21.15.100:0/400882386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af006e9c0 0x7f3af00aad20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3af0068ba0 tx=0x7f3ae8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bb0be38d2a40db server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3ae77fe700 1 -- 172.21.15.100:0/400882386 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800e070 con 0x7f3af006e9c0 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3ae77fe700 1 -- 172.21.15.100:0/400882386 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae800bab0 con 0x7f3af006e9c0 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3ae77fe700 1 -- 172.21.15.100:0/400882386 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800be70 con 0x7f3af006e9c0 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 -- 172.21.15.100:0/400882386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af006e9c0 msgr2=0x7f3af00aad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/400882386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af006e9c0 0x7f3af00aad20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3af0068ba0 tx=0x7f3ae8009780 comp rx=0 tx=0).stop 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 -- 172.21.15.100:0/400882386 shutdown_connections 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/400882386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af006e9c0 0x7f3af00aad20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/400882386 >> v2:172.21.15.123:3300/0 conn(0x7f3af0155140 0x7f3af0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 -- 172.21.15.100:0/400882386 >> 172.21.15.100:0/400882386 conn(0x7f3af0067810 msgr2=0x7f3af0067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:17.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 -- 172.21.15.100:0/400882386 shutdown_connections 2024-08-17T02:25:17.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.716+0000 7f3af6c7b700 1 -- 172.21.15.100:0/400882386 wait complete. 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 Processor -- start 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 -- start start 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af006e9c0 0x7f3af00994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3af00aed20 0x7f3af015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0155140 0x7f3af0148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00af2b0 con 0x7f3af0155140 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00af590 con 0x7f3af006e9c0 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af015a560 con 0x7f3af00aed20 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0155140 0x7f3af0148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0155140 0x7f3af0148790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50430/0 (socket says 172.21.15.100:50430) 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 -- 172.21.15.100:0/1841186280 learned_addr learned my addr 172.21.15.100:0/1841186280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af4a17700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af006e9c0 0x7f3af00994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3ae7fff700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3af00aed20 0x7f3af015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 -- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3af00aed20 msgr2=0x7f3af015a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3af00aed20 0x7f3af015a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 -- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af006e9c0 msgr2=0x7f3af00994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af006e9c0 0x7f3af00994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 -- 172.21.15.100:0/1841186280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0009420 con 0x7f3af0155140 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af5218700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0155140 0x7f3af0148790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800b610 tx=0x7f3ae8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3ae4ff9700 1 -- 172.21.15.100:0/1841186280 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8012de0 con 0x7f3af0155140 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3ae4ff9700 1 -- 172.21.15.100:0/1841186280 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae8027470 con 0x7f3af0155140 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae8009a20 con 0x7f3af0155140 2024-08-17T02:25:17.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3ae4ff9700 1 -- 172.21.15.100:0/1841186280 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8027830 con 0x7f3af0155140 2024-08-17T02:25:17.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.720+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af015a8b0 con 0x7f3af0155140 2024-08-17T02:25:17.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.724+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af015aee0 con 0x7f3af0155140 2024-08-17T02:25:17.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.724+0000 7f3ae4ff9700 1 -- 172.21.15.100:0/1841186280 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3ae8024070 con 0x7f3af0155140 2024-08-17T02:25:17.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.724+0000 7f3ae4ff9700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3adc060860 0x7f3adc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:17.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.724+0000 7f3ae4ff9700 1 -- 172.21.15.100:0/1841186280 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f3ae807f190 con 0x7f3af0155140 2024-08-17T02:25:17.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.724+0000 7f3af4a17700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3adc060860 0x7f3adc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:17.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.724+0000 7f3af4a17700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3adc060860 0x7f3adc062d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3af006e300 tx=0x7f3ae00091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:17.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.732+0000 7f3ae4ff9700 1 -- 172.21.15.100:0/1841186280 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3ae804f560 con 0x7f3af0155140 2024-08-17T02:25:17.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.932+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000351990"} v 0) v1 -- 0x7f3af00645c0 con 0x7f3af0155140 2024-08-17T02:25:17.934 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-08-17T02:25:17.932+0000 7f6fd553c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-17T02:25:17.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.936+0000 7f3ae4ff9700 1 -- 172.21.15.100:0/1841186280 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000351990"}]=0 volume_0000000000351990 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 195+0+0 (secure 0 0 0) 0x7f3ae8052b80 con 0x7f3af0155140 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3adc060860 msgr2=0x7f3adc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3adc060860 0x7f3adc062d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3af006e300 tx=0x7f3ae00091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0155140 msgr2=0x7f3af0148790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0155140 0x7f3af0148790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800b610 tx=0x7f3ae8019000 comp rx=0 tx=0).stop 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 shutdown_connections 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3adc060860 0x7f3adc062d10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0155140 0x7f3af0148790 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3af00aed20 0x7f3af015a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 --2- 172.21.15.100:0/1841186280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af006e9c0 0x7f3af00994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 >> 172.21.15.100:0/1841186280 conn(0x7f3af0067810 msgr2=0x7f3af01582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:17.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 shutdown_connections 2024-08-17T02:25:17.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:17.940+0000 7f3af6c7b700 1 -- 172.21.15.100:0/1841186280 wait complete. 2024-08-17T02:25:17.943 INFO:teuthology.orchestra.run.smithi100.stderr:volume_0000000000351990 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-17T02:25:17.946 INFO:tasks.ceph.mds.c.smithi100.stderr: -3161> 2024-08-17T02:23:07.639+0000 7fa7583d3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:25:17.956 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000351990 --yes-i-really-mean-it 2024-08-17T02:25:17.973 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-08-17T02:25:18.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.056+0000 7f8d3359e700 1 Processor -- start 2024-08-17T02:25:18.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 -- start start 2024-08-17T02:25:18.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b0ea0 0x7f8d340b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d340ae8e0 0x7f8d340b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d340b31f0 0x7f8d34069260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d34068dd0 con 0x7f8d340b31f0 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d340b3d60 con 0x7f8d340ae8e0 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340aed30 con 0x7f8d340b0ea0 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d340b31f0 0x7f8d34069260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d340b31f0 0x7f8d34069260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50456/0 (socket says 172.21.15.100:50456) 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 -- 172.21.15.100:0/2310679488 learned_addr learned my addr 172.21.15.100:0/2310679488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d29d9b700 1 --2- 172.21.15.100:0/2310679488 >> v2:172.21.15.123:3300/0 conn(0x7f8d340ae8e0 0x7f8d340b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:18.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3259c700 1 --2- 172.21.15.100:0/2310679488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b0ea0 0x7f8d340b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 -- 172.21.15.100:0/2310679488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b0ea0 msgr2=0x7f8d340b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 --2- 172.21.15.100:0/2310679488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b0ea0 0x7f8d340b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 -- 172.21.15.100:0/2310679488 >> v2:172.21.15.123:3300/0 conn(0x7f8d340ae8e0 msgr2=0x7f8d340b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 --2- 172.21.15.100:0/2310679488 >> v2:172.21.15.123:3300/0 conn(0x7f8d340ae8e0 0x7f8d340b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 -- 172.21.15.100:0/2310679488 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340b3a80 con 0x7f8d340b31f0 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d32d9d700 1 --2- 172.21.15.100:0/2310679488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b31f0 0x7f8d34069260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009f20 tx=0x7f8d2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76b98b65442f5b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d31d9b700 1 -- 172.21.15.100:0/2310679488 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00e070 con 0x7f8d340b31f0 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d31d9b700 1 -- 172.21.15.100:0/2310679488 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c00b930 con 0x7f8d340b31f0 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d31d9b700 1 -- 172.21.15.100:0/2310679488 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0092e0 con 0x7f8d340b31f0 2024-08-17T02:25:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 -- 172.21.15.100:0/2310679488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b31f0 msgr2=0x7f8d34069260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 --2- 172.21.15.100:0/2310679488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b31f0 0x7f8d34069260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009f20 tx=0x7f8d2c009840 comp rx=0 tx=0).stop 2024-08-17T02:25:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 -- 172.21.15.100:0/2310679488 shutdown_connections 2024-08-17T02:25:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.060+0000 7f8d3359e700 1 --2- 172.21.15.100:0/2310679488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b31f0 0x7f8d34069260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 --2- 172.21.15.100:0/2310679488 >> v2:172.21.15.123:3300/0 conn(0x7f8d340ae8e0 0x7f8d340b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 --2- 172.21.15.100:0/2310679488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b0ea0 0x7f8d340b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- 172.21.15.100:0/2310679488 >> 172.21.15.100:0/2310679488 conn(0x7f8d340ce920 msgr2=0x7f8d3406c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- 172.21.15.100:0/2310679488 shutdown_connections 2024-08-17T02:25:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- 172.21.15.100:0/2310679488 wait complete. 2024-08-17T02:25:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 Processor -- start 2024-08-17T02:25:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- start start 2024-08-17T02:25:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d340ae8e0 0x7f8d3413e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b0ea0 0x7f8d340c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b31f0 0x7f8d340c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3406cd20 con 0x7f8d340b0ea0 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3406b120 con 0x7f8d340ae8e0 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3413ea20 con 0x7f8d340b31f0 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b0ea0 0x7f8d340c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b0ea0 0x7f8d340c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50468/0 (socket says 172.21.15.100:50468) 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 -- 172.21.15.100:0/3797661429 learned_addr learned my addr 172.21.15.100:0/3797661429 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3259c700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d340ae8e0 0x7f8d3413e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d32d9d700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b31f0 0x7f8d340c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 -- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b31f0 msgr2=0x7f8d340c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:18.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b31f0 0x7f8d340c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 -- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d340ae8e0 msgr2=0x7f8d3413e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d340ae8e0 0x7f8d3413e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 -- 172.21.15.100:0/3797661429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c009ae0 con 0x7f8d340b0ea0 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d29d9b700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b0ea0 0x7f8d340c9a80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00ecb0 tx=0x7f8d1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d2affd700 1 -- 172.21.15.100:0/3797661429 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00c850 con 0x7f8d340b0ea0 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d2affd700 1 -- 172.21.15.100:0/3797661429 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c00c9b0 con 0x7f8d340b0ea0 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3406c3f0 con 0x7f8d340b0ea0 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d2affd700 1 -- 172.21.15.100:0/3797661429 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00cd50 con 0x7f8d340b0ea0 2024-08-17T02:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.064+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3406c9b0 con 0x7f8d340b0ea0 2024-08-17T02:25:18.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.068+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3406ced0 con 0x7f8d340b0ea0 2024-08-17T02:25:18.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.068+0000 7f8d2affd700 1 -- 172.21.15.100:0/3797661429 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8d1c013680 con 0x7f8d340b0ea0 2024-08-17T02:25:18.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.068+0000 7f8d2affd700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d18060860 0x7f8d18062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:18.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.068+0000 7f8d2affd700 1 -- 172.21.15.100:0/3797661429 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5923+0+0 (secure 0 0 0) 0x7f8d1c07cf30 con 0x7f8d340b0ea0 2024-08-17T02:25:18.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.068+0000 7f8d3259c700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d18060860 0x7f8d18062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:18.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.072+0000 7f8d3259c700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d18060860 0x7f8d18062d10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8d3406d070 tx=0x7f8d240091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:18.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.076+0000 7f8d2affd700 1 -- 172.21.15.100:0/3797661429 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8d3406ced0 con 0x7f8d340b0ea0 2024-08-17T02:25:18.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.276+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000351990", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8d3406ced0 con 0x7f8d340b0ea0 2024-08-17T02:25:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.944+0000 7f8d2affd700 1 -- 172.21.15.100:0/3797661429 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000351990", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f8d3406ced0 con 0x7f8d340b0ea0 2024-08-17T02:25:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d18060860 msgr2=0x7f8d18062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d18060860 0x7f8d18062d10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8d3406d070 tx=0x7f8d240091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b0ea0 msgr2=0x7f8d340c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b0ea0 0x7f8d340c9a80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00ecb0 tx=0x7f8d1c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 shutdown_connections 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8d18060860 0x7f8d18062d10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d340b31f0 0x7f8d340c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d340b0ea0 0x7f8d340c9a80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 --2- 172.21.15.100:0/3797661429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d340ae8e0 0x7f8d3413e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 >> 172.21.15.100:0/3797661429 conn(0x7f8d340ce920 msgr2=0x7f8d340af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:18.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 shutdown_connections 2024-08-17T02:25:18.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:18.948+0000 7f8d3359e700 1 -- 172.21.15.100:0/3797661429 wait complete. 2024-08-17T02:25:18.964 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000351990.meta cephfs.volume_0000000000351990.meta --yes-i-really-really-mean-it 2024-08-17T02:25:19.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.128+0000 7fc3a3fff700 1 Processor -- start 2024-08-17T02:25:19.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.128+0000 7fc3a3fff700 1 -- start start 2024-08-17T02:25:19.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a40b29c0 0x7fc3a4157240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:19.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a3fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40cb590 con 0x7fc3a40aed20 2024-08-17T02:25:19.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a3fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40caa00 con 0x7fc3a4156e70 2024-08-17T02:25:19.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40b1390 con 0x7fc3a40b29c0 2024-08-17T02:25:19.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a37fe700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc3a40aed20 0x7fc3a40cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58308/0 (socket says 172.21.15.100:58308) 2024-08-17T02:25:19.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a37fe700 1 -- 172.21.15.100:0/2088645578 learned_addr learned my addr 172.21.15.100:0/2088645578 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:19.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a27fc700 1 --2- 172.21.15.100:0/2088645578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a40b29c0 0x7fc3a4157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:19.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a1ffb700 1 -- 172.21.15.100:0/2088645578 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301310384 0 0) 0x7fc3a40cb590 con 0x7fc3a40aed20 2024-08-17T02:25:19.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a1ffb700 1 -- 172.21.15.100:0/2088645578 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc38c003570 con 0x7fc3a40aed20 2024-08-17T02:25:19.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a1ffb700 1 -- 172.21.15.100:0/2088645578 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994962166 0 0) 0x7fc3a40caa00 con 0x7fc3a4156e70 2024-08-17T02:25:19.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a1ffb700 1 -- 172.21.15.100:0/2088645578 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a40cb590 con 0x7fc3a4156e70 2024-08-17T02:25:19.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a27fc700 1 -- 172.21.15.100:0/2088645578 >> v1:172.21.15.123:6789/0 conn(0x7fc3a4156e70 legacy=0x7fc3a40b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:19.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a27fc700 1 -- 172.21.15.100:0/2088645578 >> v1:172.21.15.100:6789/0 conn(0x7fc3a40aed20 legacy=0x7fc3a40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:19.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a27fc700 1 -- 172.21.15.100:0/2088645578 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40caca0 con 0x7fc3a40b29c0 2024-08-17T02:25:19.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a27fc700 1 --2- 172.21.15.100:0/2088645578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a40b29c0 0x7fc3a4157240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc398009910 tx=0x7fc3980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27299c171ce78f1e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:19.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a1ffb700 1 -- 172.21.15.100:0/2088645578 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398011630 con 0x7fc3a40b29c0 2024-08-17T02:25:19.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a1ffb700 1 -- 172.21.15.100:0/2088645578 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc398011790 con 0x7fc3a40b29c0 2024-08-17T02:25:19.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.132+0000 7fc3a1ffb700 1 -- 172.21.15.100:0/2088645578 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398011b10 con 0x7fc3a40b29c0 2024-08-17T02:25:19.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.136+0000 7fc3a3fff700 1 -- 172.21.15.100:0/2088645578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a40b29c0 msgr2=0x7fc3a4157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:19.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.136+0000 7fc3a3fff700 1 --2- 172.21.15.100:0/2088645578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a40b29c0 0x7fc3a4157240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc398009910 tx=0x7fc3980091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:19.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.136+0000 7fc3a3fff700 1 -- 172.21.15.100:0/2088645578 shutdown_connections 2024-08-17T02:25:19.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.136+0000 7fc3a3fff700 1 --2- 172.21.15.100:0/2088645578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a40b29c0 0x7fc3a4157240 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:19.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.136+0000 7fc3a3fff700 1 -- 172.21.15.100:0/2088645578 >> 172.21.15.100:0/2088645578 conn(0x7fc3a40cd8d0 msgr2=0x7fc3a406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:19.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.136+0000 7fc3a3fff700 1 -- 172.21.15.100:0/2088645578 shutdown_connections 2024-08-17T02:25:19.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.136+0000 7fc3a3fff700 1 -- 172.21.15.100:0/2088645578 wait complete. 2024-08-17T02:25:19.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a3fff700 1 Processor -- start 2024-08-17T02:25:19.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a3fff700 1 -- start start 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40aed20 0x7fc3a413c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3a40b29c0 0x7fc3a40a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a4156e70 0x7fc3a40a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a3fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40a71e0 con 0x7fc3a40aed20 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40a6140 con 0x7fc3a40b29c0 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40a6420 con 0x7fc3a4156e70 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40aed20 0x7fc3a413c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40aed20 0x7fc3a413c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50478/0 (socket says 172.21.15.100:50478) 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 -- 172.21.15.100:0/3016593387 learned_addr learned my addr 172.21.15.100:0/3016593387 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:19.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a37fe700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a4156e70 0x7fc3a40a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:19.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a27fc700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3a40b29c0 0x7fc3a40a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 -- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a4156e70 msgr2=0x7fc3a40a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a4156e70 0x7fc3a40a7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 -- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3a40b29c0 msgr2=0x7fc3a40a7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3a40b29c0 0x7fc3a40a7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 -- 172.21.15.100:0/3016593387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398009420 con 0x7fc3a40aed20 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a37fe700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a4156e70 0x7fc3a40a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.140+0000 7fc3a2ffd700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40aed20 0x7fc3a413c6e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc39400c580 tx=0x7fc394007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc38affd700 1 -- 172.21.15.100:0/3016593387 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394010510 con 0x7fc3a40aed20 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a40a6700 con 0x7fc3a40aed20 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc38affd700 1 -- 172.21.15.100:0/3016593387 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394010670 con 0x7fc3a40aed20 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a40a6cc0 con 0x7fc3a40aed20 2024-08-17T02:25:19.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc38affd700 1 -- 172.21.15.100:0/3016593387 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940109f0 con 0x7fc3a40aed20 2024-08-17T02:25:19.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a40a6eb0 con 0x7fc3a40aed20 2024-08-17T02:25:19.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc38affd700 1 -- 172.21.15.100:0/3016593387 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc394010b50 con 0x7fc3a40aed20 2024-08-17T02:25:19.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc38affd700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc390060860 0x7fc390062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:19.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc38affd700 1 -- 172.21.15.100:0/3016593387 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5923+0+0 (secure 0 0 0) 0x7fc39407c670 con 0x7fc3a40aed20 2024-08-17T02:25:19.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.144+0000 7fc3a27fc700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc390060860 0x7fc390062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:19.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.148+0000 7fc3a27fc700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc390060860 0x7fc390062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40b2dd0 tx=0x7fc398017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:19.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.152+0000 7fc38affd700 1 -- 172.21.15.100:0/3016593387 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc39404c9e0 con 0x7fc3a40aed20 2024-08-17T02:25:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.316+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000351990.meta", "pool2": "cephfs.volume_0000000000351990.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc3a40644b0 con 0x7fc3a40aed20 2024-08-17T02:25:19.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.952+0000 7fc38affd700 1 -- 172.21.15.100:0/3016593387 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000351990.meta", "pool2": "cephfs.volume_0000000000351990.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000351990.meta' removed v43) v1 ==== 241+0+0 (secure 0 0 0) 0x7fc394050000 con 0x7fc3a40aed20 2024-08-17T02:25:19.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc390060860 msgr2=0x7fc390062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:19.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc390060860 0x7fc390062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40b2dd0 tx=0x7fc398017000 comp rx=0 tx=0).stop 2024-08-17T02:25:19.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40aed20 msgr2=0x7fc3a413c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40aed20 0x7fc3a413c6e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc39400c580 tx=0x7fc394007e10 comp rx=0 tx=0).stop 2024-08-17T02:25:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 shutdown_connections 2024-08-17T02:25:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fc390060860 0x7fc390062d10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3a4156e70 0x7fc3a40a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3a40b29c0 0x7fc3a40a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 --2- 172.21.15.100:0/3016593387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40aed20 0x7fc3a413c6e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 >> 172.21.15.100:0/3016593387 conn(0x7fc3a40cd8d0 msgr2=0x7fc3a4155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:19.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 shutdown_connections 2024-08-17T02:25:19.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:19.956+0000 7fc3a3fff700 1 -- 172.21.15.100:0/3016593387 wait complete. 2024-08-17T02:25:19.961 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs.volume_0000000000351990.meta' removed 2024-08-17T02:25:19.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000351990.data cephfs.volume_0000000000351990.data --yes-i-really-really-mean-it 2024-08-17T02:25:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.084+0000 7f27e5fe3700 1 Processor -- start 2024-08-17T02:25:20.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.084+0000 7f27e5fe3700 1 -- start start 2024-08-17T02:25:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f27e00cd2b0 0x7f27e00cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27e00caeb0 0x7f27e00ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00cb300 0x7f27e00ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e00cc480 con 0x7f27e00caeb0 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e0159720 con 0x7f27e00cd2b0 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00cb870 con 0x7f27e00cb300 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27e00caeb0 0x7f27e00ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27e00caeb0 0x7f27e00ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50490/0 (socket says 172.21.15.100:50490) 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 -- 172.21.15.100:0/3222563714 learned_addr learned my addr 172.21.15.100:0/3222563714 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27dffff700 1 --2- 172.21.15.100:0/3222563714 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00cb300 0x7f27e00ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27df7fe700 1 --2- 172.21.15.100:0/3222563714 >> v2:172.21.15.123:3300/0 conn(0x7f27e00cd2b0 0x7f27e00cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:20.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 -- 172.21.15.100:0/3222563714 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00cb300 msgr2=0x7f27e00ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:20.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 --2- 172.21.15.100:0/3222563714 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00cb300 0x7f27e00ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 -- 172.21.15.100:0/3222563714 >> v2:172.21.15.123:3300/0 conn(0x7f27e00cd2b0 msgr2=0x7f27e00cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:20.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 --2- 172.21.15.100:0/3222563714 >> v2:172.21.15.123:3300/0 conn(0x7f27e00cd2b0 0x7f27e00cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 -- 172.21.15.100:0/3222563714 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00cbb50 con 0x7f27e00caeb0 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27deffd700 1 --2- 172.21.15.100:0/3222563714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00caeb0 0x7f27e00ce460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f27e008a610 tx=0x7f27d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe748dfdb25ddc91 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27de7fc700 1 -- 172.21.15.100:0/3222563714 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d000b640 con 0x7f27e00caeb0 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27de7fc700 1 -- 172.21.15.100:0/3222563714 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d000b7a0 con 0x7f27e00caeb0 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27de7fc700 1 -- 172.21.15.100:0/3222563714 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d000bb60 con 0x7f27e00caeb0 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 -- 172.21.15.100:0/3222563714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00caeb0 msgr2=0x7f27e00ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/3222563714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00caeb0 0x7f27e00ce460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f27e008a610 tx=0x7f27d0009250 comp rx=0 tx=0).stop 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 -- 172.21.15.100:0/3222563714 shutdown_connections 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/3222563714 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00cb300 0x7f27e00ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/3222563714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00caeb0 0x7f27e00ce460 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/3222563714 >> v2:172.21.15.123:3300/0 conn(0x7f27e00cd2b0 0x7f27e00cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 -- 172.21.15.100:0/3222563714 >> 172.21.15.100:0/3222563714 conn(0x7f27e0099550 msgr2=0x7f27e0067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 -- 172.21.15.100:0/3222563714 shutdown_connections 2024-08-17T02:25:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.088+0000 7f27e5fe3700 1 -- 172.21.15.100:0/3222563714 wait complete. 2024-08-17T02:25:20.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 Processor -- start 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 -- start start 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00caeb0 0x7f27e0158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e00cb300 0x7f27e00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00cd2b0 0x7f27e00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e01458a0 con 0x7f27e00cd2b0 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e0145b30 con 0x7f27e00cb300 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e0146990 con 0x7f27e00caeb0 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27dffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00cd2b0 0x7f27e00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27df7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00caeb0 0x7f27e0158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27df7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00caeb0 0x7f27e0158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.100:50306/0 (socket says 172.21.15.100:50306) 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27df7fe700 1 -- 172.21.15.100:0/2933785511 learned_addr learned my addr 172.21.15.100:0/2933785511 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27deffd700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e00cb300 0x7f27e00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27dffff700 1 -- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00caeb0 msgr2=0x7f27e0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27dffff700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00caeb0 0x7f27e0158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27dffff700 1 -- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e00cb300 msgr2=0x7f27e00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27dffff700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e00cb300 0x7f27e00c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27dffff700 1 -- 172.21.15.100:0/2933785511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d00094f0 con 0x7f27e00cd2b0 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27df7fe700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00caeb0 0x7f27e0158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27dffff700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00cd2b0 0x7f27e00c9ef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ecc0 tx=0x7f27d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27c77fe700 1 -- 172.21.15.100:0/2933785511 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d800c8c0 con 0x7f27e00cd2b0 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27c77fe700 1 -- 172.21.15.100:0/2933785511 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d800f040 con 0x7f27e00cd2b0 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e0145d80 con 0x7f27e00cd2b0 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27c77fe700 1 -- 172.21.15.100:0/2933785511 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d800cc90 con 0x7f27e00cd2b0 2024-08-17T02:25:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.092+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e0146340 con 0x7f27e00cd2b0 2024-08-17T02:25:20.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.096+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e0146b00 con 0x7f27e00cd2b0 2024-08-17T02:25:20.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.096+0000 7f27c77fe700 1 -- 172.21.15.100:0/2933785511 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f27d8014480 con 0x7f27e00cd2b0 2024-08-17T02:25:20.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.096+0000 7f27c77fe700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27cc060860 0x7f27cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.096+0000 7f27c77fe700 1 -- 172.21.15.100:0/2933785511 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5448+0+0 (secure 0 0 0) 0x7f27d807d600 con 0x7f27e00cd2b0 2024-08-17T02:25:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.096+0000 7f27df7fe700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27cc060860 0x7f27cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.096+0000 7f27df7fe700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27cc060860 0x7f27cc062d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009fd0 tx=0x7f27d40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:20.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.104+0000 7f27c77fe700 1 -- 172.21.15.100:0/2933785511 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f27d804dab0 con 0x7f27e00cd2b0 2024-08-17T02:25:20.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.276+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000351990.data", "pool2": "cephfs.volume_0000000000351990.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f27e00638f0 con 0x7f27e00cd2b0 2024-08-17T02:25:20.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.960+0000 7f27c77fe700 1 -- 172.21.15.100:0/2933785511 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000351990.data", "pool2": "cephfs.volume_0000000000351990.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000351990.data' removed v44) v1 ==== 241+0+0 (secure 0 0 0) 0x7f27d80510d0 con 0x7f27e00cd2b0 2024-08-17T02:25:20.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27cc060860 msgr2=0x7f27cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27cc060860 0x7f27cc062d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009fd0 tx=0x7f27d40091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00cd2b0 msgr2=0x7f27e00c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00cd2b0 0x7f27e00c9ef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ecc0 tx=0x7f27d800b040 comp rx=0 tx=0).stop 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 shutdown_connections 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f27cc060860 0x7f27cc062d10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e00cd2b0 0x7f27e00c9ef0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e00cb300 0x7f27e00c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 --2- 172.21.15.100:0/2933785511 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e00caeb0 0x7f27e0158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 >> 172.21.15.100:0/2933785511 conn(0x7f27e0099550 msgr2=0x7f27e00ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 shutdown_connections 2024-08-17T02:25:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:20.968+0000 7f27e5fe3700 1 -- 172.21.15.100:0/2933785511 wait complete. 2024-08-17T02:25:20.972 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs.volume_0000000000351990.data' removed 2024-08-17T02:25:20.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-08-17T02:25:21.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.096+0000 7fe68856b700 1 Processor -- start 2024-08-17T02:25:21.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.096+0000 7fe68856b700 1 -- start start 2024-08-17T02:25:21.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe680159400 0x7fe6800b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe68006bac0 0x7fe6800686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6800adb00 con 0x7fe6801597d0 2024-08-17T02:25:21.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68006bf60 con 0x7fe68006bac0 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6800ac630 con 0x7fe680159400 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686b08700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe68006bac0 0x7fe6800686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686b08700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe68006bac0 0x7fe6800686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50510/0 (socket says 172.21.15.100:50510) 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686b08700 1 -- 172.21.15.100:0/198776546 learned_addr learned my addr 172.21.15.100:0/198776546 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686307700 1 --2- 172.21.15.100:0/198776546 >> v2:172.21.15.123:3300/0 conn(0x7fe680159400 0x7fe6800b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686b08700 1 -- 172.21.15.100:0/198776546 >> v2:172.21.15.123:3300/0 conn(0x7fe680159400 msgr2=0x7fe6800b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686b08700 1 --2- 172.21.15.100:0/198776546 >> v2:172.21.15.123:3300/0 conn(0x7fe680159400 0x7fe6800b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686b08700 1 -- 172.21.15.100:0/198776546 >> v1:172.21.15.123:6789/0 conn(0x7fe6801597d0 legacy=0x7fe6800ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686b08700 1 -- 172.21.15.100:0/198776546 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6800ac910 con 0x7fe68006bac0 2024-08-17T02:25:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe686b08700 1 --2- 172.21.15.100:0/198776546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68006bac0 0x7fe6800686a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe6800701a0 tx=0x7fe678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2000a43f97805a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe685305700 1 -- 172.21.15.100:0/198776546 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678009e70 con 0x7fe68006bac0 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe685305700 1 -- 172.21.15.100:0/198776546 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67800b860 con 0x7fe68006bac0 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe685305700 1 -- 172.21.15.100:0/198776546 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800bc00 con 0x7fe68006bac0 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 -- 172.21.15.100:0/198776546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68006bac0 msgr2=0x7fe6800686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 --2- 172.21.15.100:0/198776546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68006bac0 0x7fe6800686a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe6800701a0 tx=0x7fe678009250 comp rx=0 tx=0).stop 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 -- 172.21.15.100:0/198776546 shutdown_connections 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 --2- 172.21.15.100:0/198776546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68006bac0 0x7fe6800686a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.100+0000 7fe68856b700 1 --2- 172.21.15.100:0/198776546 >> v2:172.21.15.123:3300/0 conn(0x7fe680159400 0x7fe6800b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.104+0000 7fe68856b700 1 -- 172.21.15.100:0/198776546 >> 172.21.15.100:0/198776546 conn(0x7fe6800c9bf0 msgr2=0x7fe68006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.104+0000 7fe68856b700 1 -- 172.21.15.100:0/198776546 shutdown_connections 2024-08-17T02:25:21.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.104+0000 7fe68856b700 1 -- 172.21.15.100:0/198776546 wait complete. 2024-08-17T02:25:21.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.104+0000 7fe68856b700 1 Processor -- start 2024-08-17T02:25:21.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.104+0000 7fe68856b700 1 -- start start 2024-08-17T02:25:21.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006bac0 0x7fe68013a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe680159400 0x7fe6801468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6801597d0 0x7fe68014f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68013a690 con 0x7fe680159400 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe680146420 con 0x7fe6801597d0 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe680146df0 con 0x7fe68006bac0 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe680159400 0x7fe6801468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe680159400 0x7fe6801468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50526/0 (socket says 172.21.15.100:50526) 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 -- 172.21.15.100:0/2226389608 learned_addr learned my addr 172.21.15.100:0/2226389608 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe686b08700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6801597d0 0x7fe68014f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe686307700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006bac0 0x7fe68013a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 -- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006bac0 msgr2=0x7fe68013a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006bac0 0x7fe68013a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 -- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6801597d0 msgr2=0x7fe68014f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6801597d0 0x7fe68014f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 -- 172.21.15.100:0/2226389608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780094f0 con 0x7fe680159400 2024-08-17T02:25:21.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe686b08700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6801597d0 0x7fe68014f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:21.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe685b06700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe680159400 0x7fe6801468b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe67400c790 tx=0x7fe674007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:21.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe67e7fc700 1 -- 172.21.15.100:0/2226389608 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674011590 con 0x7fe680159400 2024-08-17T02:25:21.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe67e7fc700 1 -- 172.21.15.100:0/2226389608 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe674011710 con 0x7fe680159400 2024-08-17T02:25:21.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68014fa80 con 0x7fe680159400 2024-08-17T02:25:21.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe67e7fc700 1 -- 172.21.15.100:0/2226389608 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674011ad0 con 0x7fe680159400 2024-08-17T02:25:21.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe680145af0 con 0x7fe680159400 2024-08-17T02:25:21.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6800645c0 con 0x7fe680159400 2024-08-17T02:25:21.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.108+0000 7fe67e7fc700 1 -- 172.21.15.100:0/2226389608 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe67401b3b0 con 0x7fe680159400 2024-08-17T02:25:21.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.112+0000 7fe67e7fc700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe66c060890 0x7fe66c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.112+0000 7fe67e7fc700 1 -- 172.21.15.100:0/2226389608 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe67407bca0 con 0x7fe680159400 2024-08-17T02:25:21.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.112+0000 7fe686307700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe66c060890 0x7fe66c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.112+0000 7fe686307700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe66c060890 0x7fe66c062d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe67000f570 tx=0x7fe67000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.116+0000 7fe67e7fc700 1 -- 172.21.15.100:0/2226389608 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe67404c390 con 0x7fe680159400 2024-08-17T02:25:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.276+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe680146560 con 0x7fe680159400 2024-08-17T02:25:21.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe67e7fc700 1 -- 172.21.15.100:0/2226389608 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe6740170a0 con 0x7fe680159400 2024-08-17T02:25:21.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe66c060890 msgr2=0x7fe66c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe66c060890 0x7fe66c062d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe67000f570 tx=0x7fe67000b040 comp rx=0 tx=0).stop 2024-08-17T02:25:21.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe680159400 msgr2=0x7fe6801468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe680159400 0x7fe6801468b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe67400c790 tx=0x7fe674007f40 comp rx=0 tx=0).stop 2024-08-17T02:25:21.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 shutdown_connections 2024-08-17T02:25:21.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe66c060890 0x7fe66c062d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6801597d0 0x7fe68014f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe680159400 0x7fe6801468b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 --2- 172.21.15.100:0/2226389608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006bac0 0x7fe68013a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 >> 172.21.15.100:0/2226389608 conn(0x7fe6800c9bf0 msgr2=0x7fe6800ab970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:21.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 shutdown_connections 2024-08-17T02:25:21.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.284+0000 7fe68856b700 1 -- 172.21.15.100:0/2226389608 wait complete. 2024-08-17T02:25:21.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-17T02:25:21.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.404+0000 7f868683f700 1 Processor -- start 2024-08-17T02:25:21.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.404+0000 7f868683f700 1 -- start start 2024-08-17T02:25:21.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8680153a50 0x7f86800687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86800ae000 0x7f868006d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868015ce60 con 0x7f86800ae430 2024-08-17T02:25:21.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868015d0b0 con 0x7f8680153a50 2024-08-17T02:25:21.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868015d390 con 0x7f86800ae000 2024-08-17T02:25:21.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867ffff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8680153a50 0x7f86800687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867ffff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8680153a50 0x7f86800687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50542/0 (socket says 172.21.15.100:50542) 2024-08-17T02:25:21.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867ffff700 1 -- 172.21.15.100:0/1652759128 learned_addr learned my addr 172.21.15.100:0/1652759128 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:21.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867f7fe700 1 --2- 172.21.15.100:0/1652759128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86800ae000 0x7f868006d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867ffff700 1 -- 172.21.15.100:0/1652759128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86800ae000 msgr2=0x7f868006d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867ffff700 1 --2- 172.21.15.100:0/1652759128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86800ae000 0x7f868006d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867ffff700 1 -- 172.21.15.100:0/1652759128 >> v1:172.21.15.123:6789/0 conn(0x7f86800ae430 legacy=0x7f86800afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867effd700 1 -- 172.21.15.100:0/1652759128 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680291146 0 0) 0x7f868015ce60 con 0x7f86800ae430 2024-08-17T02:25:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867ffff700 1 -- 172.21.15.100:0/1652759128 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868015d670 con 0x7f8680153a50 2024-08-17T02:25:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867ffff700 1 --2- 172.21.15.100:0/1652759128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680153a50 0x7f86800687a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f86740099b0 tx=0x7f8674009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1230c7ba38897ae server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867effd700 1 -- 172.21.15.100:0/1652759128 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674009ea0 con 0x7f8680153a50 2024-08-17T02:25:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867effd700 1 -- 172.21.15.100:0/1652759128 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867400b860 con 0x7f8680153a50 2024-08-17T02:25:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f867effd700 1 -- 172.21.15.100:0/1652759128 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400bc00 con 0x7f8680153a50 2024-08-17T02:25:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 -- 172.21.15.100:0/1652759128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680153a50 msgr2=0x7f86800687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 --2- 172.21.15.100:0/1652759128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680153a50 0x7f86800687a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f86740099b0 tx=0x7f8674009250 comp rx=0 tx=0).stop 2024-08-17T02:25:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 -- 172.21.15.100:0/1652759128 shutdown_connections 2024-08-17T02:25:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 --2- 172.21.15.100:0/1652759128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86800ae000 0x7f868006d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 --2- 172.21.15.100:0/1652759128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680153a50 0x7f86800687a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 -- 172.21.15.100:0/1652759128 >> 172.21.15.100:0/1652759128 conn(0x7f86800ceb30 msgr2=0x7f868006cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 -- 172.21.15.100:0/1652759128 shutdown_connections 2024-08-17T02:25:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.408+0000 7f868683f700 1 -- 172.21.15.100:0/1652759128 wait complete. 2024-08-17T02:25:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f868683f700 1 Processor -- start 2024-08-17T02:25:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f868683f700 1 -- start start 2024-08-17T02:25:21.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f868683f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86800ae000 0x7f86800cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f868683f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86800ae430 0x7f8680149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f868683f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8680153a50 0x7f8680156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f868683f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86801544a0 con 0x7f86800ae000 2024-08-17T02:25:21.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f868683f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800c97a0 con 0x7f86800ae430 2024-08-17T02:25:21.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f868683f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86800c9a80 con 0x7f8680153a50 2024-08-17T02:25:21.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86800ae000 0x7f86800cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86800ae000 0x7f86800cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50552/0 (socket says 172.21.15.100:50552) 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867ffff700 1 -- 172.21.15.100:0/1910380477 learned_addr learned my addr 172.21.15.100:0/1910380477 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867f7fe700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86800ae430 0x7f8680149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f8684ddc700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8680153a50 0x7f8680156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867ffff700 1 -- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8680153a50 msgr2=0x7f8680156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867ffff700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8680153a50 0x7f8680156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867ffff700 1 -- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86800ae430 msgr2=0x7f8680149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867ffff700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86800ae430 0x7f8680149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.412+0000 7f867ffff700 1 -- 172.21.15.100:0/1910380477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740094f0 con 0x7f86800ae000 2024-08-17T02:25:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.416+0000 7f867ffff700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86800ae000 0x7f86800cb0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f86800b40d0 tx=0x7f8674017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:21.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.416+0000 7f8667fff700 1 -- 172.21.15.100:0/1910380477 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674000d10 con 0x7f86800ae000 2024-08-17T02:25:21.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.416+0000 7f8667fff700 1 -- 172.21.15.100:0/1910380477 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867400bbe0 con 0x7f86800ae000 2024-08-17T02:25:21.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.416+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8680153e50 con 0x7f86800ae000 2024-08-17T02:25:21.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.416+0000 7f8667fff700 1 -- 172.21.15.100:0/1910380477 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674011690 con 0x7f86800ae000 2024-08-17T02:25:21.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.416+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8680149f70 con 0x7f86800ae000 2024-08-17T02:25:21.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.416+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86800645c0 con 0x7f86800ae000 2024-08-17T02:25:21.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.416+0000 7f8667fff700 1 -- 172.21.15.100:0/1910380477 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8674011ac0 con 0x7f86800ae000 2024-08-17T02:25:21.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.420+0000 7f8667fff700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f866c060860 0x7f866c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:21.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.420+0000 7f8667fff700 1 -- 172.21.15.100:0/1910380477 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f867407e170 con 0x7f86800ae000 2024-08-17T02:25:21.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.420+0000 7f867f7fe700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f866c060860 0x7f866c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:21.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.420+0000 7f867f7fe700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f866c060860 0x7f866c062d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f86800ccfc0 tx=0x7f867000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:21.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.424+0000 7f8667fff700 1 -- 172.21.15.100:0/1910380477 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f867404e7c0 con 0x7f86800ae000 2024-08-17T02:25:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.648+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f86800cd4d0 con 0x7f86800ae000 2024-08-17T02:25:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.964+0000 7f8667fff700 1 -- 172.21.15.100:0/1910380477 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v149) v1 ==== 135+0+0 (secure 0 0 0) 0x7f8674051de0 con 0x7f86800ae000 2024-08-17T02:25:21.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.968+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f866c060860 msgr2=0x7f866c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.968+0000 7f868683f700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f866c060860 0x7f866c062d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f86800ccfc0 tx=0x7f867000b040 comp rx=0 tx=0).stop 2024-08-17T02:25:21.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.968+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86800ae000 msgr2=0x7f86800cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.968+0000 7f868683f700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86800ae000 0x7f86800cb0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f86800b40d0 tx=0x7f8674017000 comp rx=0 tx=0).stop 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.972+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 shutdown_connections 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.972+0000 7f868683f700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f866c060860 0x7f866c062d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.972+0000 7f868683f700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8680153a50 0x7f8680156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.972+0000 7f868683f700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86800ae430 0x7f8680149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.972+0000 7f868683f700 1 --2- 172.21.15.100:0/1910380477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86800ae000 0x7f86800cb0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.972+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 >> 172.21.15.100:0/1910380477 conn(0x7f86800ceb30 msgr2=0x7f86800b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.972+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 shutdown_connections 2024-08-17T02:25:21.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:21.972+0000 7f868683f700 1 -- 172.21.15.100:0/1910380477 wait complete. 2024-08-17T02:25:21.994 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ERROR 2024-08-17T02:25:21.994 INFO:tasks.cephfs_test_runner: 2024-08-17T02:25:21.994 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-17T02:25:21.994 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:25:21.994 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-17T02:25:21.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi100 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000351990 --yes-i-really-mean-it' 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner: 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner:Ran 1 test in 165.242s 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner: 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner: 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-17T02:25:21.996 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-17T02:25:21.997 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-17T02:25:21.998 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-17T02:25:21.998 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-17T02:25:21.998 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-17T02:25:21.998 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-17T02:25:21.998 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-17T02:25:21.998 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-17T02:25:21.998 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi100 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000351990 --yes-i-really-mean-it' 2024-08-17T02:25:21.998 INFO:tasks.cephfs_test_runner: 2024-08-17T02:25:21.998 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:25:22.220 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=059d8cab2ce740ccb5704f2ae520bc53 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:25:22.221 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-08-17T02:25:22.230 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-08-17T02:25:22.237 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-08-17T02:25:22.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:22.237 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-08-17T02:25:22.253 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-17T02:25:22.253 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-17T02:25:22.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:22.254 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-17T02:25:22.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:22.312 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-17T02:25:22.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:22.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-08-17T02:25:22.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:22.474 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-08-17T02:25:22.481 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-17T02:25:22.482 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-17T02:25:22.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:22.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-17T02:25:22.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:22.540 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-17T02:25:22.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:25:22.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-08-17T02:25:22.688 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-17T02:25:22.698 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-08-17T02:25:22.698 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:25:22.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.808+0000 7f820e298700 1 Processor -- start 2024-08-17T02:25:22.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.808+0000 7f820e298700 1 -- start start 2024-08-17T02:25:22.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.808+0000 7f820e298700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b0d20 0x7f820806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.808+0000 7f820e298700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8208141a00 con 0x7f820806e450 2024-08-17T02:25:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.808+0000 7f820e298700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820806e8a0 con 0x7f82080b1800 2024-08-17T02:25:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.808+0000 7f820e298700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8208140b10 con 0x7f82080b0d20 2024-08-17T02:25:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820ca95700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f820806e450 0x7f82080b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58318/0 (socket says 172.21.15.100:58318) 2024-08-17T02:25:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820ca95700 1 -- 172.21.15.100:0/4257907756 learned_addr learned my addr 172.21.15.100:0/4257907756 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820d296700 1 --2- 172.21.15.100:0/4257907756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b0d20 0x7f820806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f8207fff700 1 -- 172.21.15.100:0/4257907756 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471941302 0 0) 0x7f8208141a00 con 0x7f820806e450 2024-08-17T02:25:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f8207fff700 1 -- 172.21.15.100:0/4257907756 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f4003570 con 0x7f820806e450 2024-08-17T02:25:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f8207fff700 1 -- 172.21.15.100:0/4257907756 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728551122 0 0) 0x7f820806e8a0 con 0x7f82080b1800 2024-08-17T02:25:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f8207fff700 1 -- 172.21.15.100:0/4257907756 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8208141a00 con 0x7f82080b1800 2024-08-17T02:25:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820d296700 1 -- 172.21.15.100:0/4257907756 >> v1:172.21.15.123:6789/0 conn(0x7f82080b1800 legacy=0x7f8208141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820d296700 1 -- 172.21.15.100:0/4257907756 >> v1:172.21.15.100:6789/0 conn(0x7f820806e450 legacy=0x7f82080b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820d296700 1 -- 172.21.15.100:0/4257907756 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8208140df0 con 0x7f82080b0d20 2024-08-17T02:25:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820d296700 1 --2- 172.21.15.100:0/4257907756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b0d20 0x7f820806df10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009e90 tx=0x7f81f0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39acef1eaa5aba6c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f8207fff700 1 -- 172.21.15.100:0/4257907756 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f000e070 con 0x7f82080b0d20 2024-08-17T02:25:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f8207fff700 1 -- 172.21.15.100:0/4257907756 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f000bb80 con 0x7f82080b0d20 2024-08-17T02:25:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f8207fff700 1 -- 172.21.15.100:0/4257907756 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f0012430 con 0x7f82080b0d20 2024-08-17T02:25:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820e298700 1 -- 172.21.15.100:0/4257907756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b0d20 msgr2=0x7f820806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820e298700 1 --2- 172.21.15.100:0/4257907756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b0d20 0x7f820806df10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009e90 tx=0x7f81f0009780 comp rx=0 tx=0).stop 2024-08-17T02:25:22.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820e298700 1 -- 172.21.15.100:0/4257907756 shutdown_connections 2024-08-17T02:25:22.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820e298700 1 --2- 172.21.15.100:0/4257907756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b0d20 0x7f820806df10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:22.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820e298700 1 -- 172.21.15.100:0/4257907756 >> 172.21.15.100:0/4257907756 conn(0x7f82080678d0 msgr2=0x7f8208067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:22.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820e298700 1 -- 172.21.15.100:0/4257907756 shutdown_connections 2024-08-17T02:25:22.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.812+0000 7f820e298700 1 -- 172.21.15.100:0/4257907756 wait complete. 2024-08-17T02:25:22.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 Processor -- start 2024-08-17T02:25:22.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 -- start start 2024-08-17T02:25:22.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820806e450 0x7f820816d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:22.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b0d20 0x7f82080a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:22.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b1800 0x7f8208146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:22.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080a79f0 con 0x7f82080b0d20 2024-08-17T02:25:22.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820816dcf0 con 0x7f820806e450 2024-08-17T02:25:22.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82080a6de0 con 0x7f82080b1800 2024-08-17T02:25:22.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b0d20 0x7f82080a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:22.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b0d20 0x7f82080a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50566/0 (socket says 172.21.15.100:50566) 2024-08-17T02:25:22.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 -- 172.21.15.100:0/2412408547 learned_addr learned my addr 172.21.15.100:0/2412408547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:22.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820d296700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820806e450 0x7f820816d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:22.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820da97700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b1800 0x7f8208146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:22.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 -- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b1800 msgr2=0x7f8208146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:22.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b1800 0x7f8208146570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:22.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 -- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820806e450 msgr2=0x7f820816d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:22.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820806e450 0x7f820816d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:22.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 -- 172.21.15.100:0/2412408547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0009a20 con 0x7f82080b0d20 2024-08-17T02:25:22.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820ca95700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b0d20 0x7f82080a6020 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f81f800c790 tx=0x7f81f8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:22.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f82057fa700 1 -- 172.21.15.100:0/2412408547 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8005b30 con 0x7f82080b0d20 2024-08-17T02:25:22.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f82057fa700 1 -- 172.21.15.100:0/2412408547 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8005c90 con 0x7f82080b0d20 2024-08-17T02:25:22.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82080a70c0 con 0x7f82080b0d20 2024-08-17T02:25:22.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f82057fa700 1 -- 172.21.15.100:0/2412408547 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8007300 con 0x7f82080b0d20 2024-08-17T02:25:22.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.816+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82080a7680 con 0x7f82080b0d20 2024-08-17T02:25:22.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.820+0000 7f82057fa700 1 -- 172.21.15.100:0/2412408547 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f81f801b3b0 con 0x7f82080b0d20 2024-08-17T02:25:22.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.820+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82080cace0 con 0x7f82080b0d20 2024-08-17T02:25:22.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.820+0000 7f82057fa700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f81e8060860 0x7f81e8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:22.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.820+0000 7f820d296700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f81e8060860 0x7f81e8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:22.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.820+0000 7f82057fa700 1 -- 172.21.15.100:0/2412408547 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f81f807be50 con 0x7f82080b0d20 2024-08-17T02:25:22.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.820+0000 7f820d296700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f81e8060860 0x7f81e8062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009de0 tx=0x7f81f0019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:22.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.828+0000 7f82057fa700 1 -- 172.21.15.100:0/2412408547 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f81f804ff30 con 0x7f82080b0d20 2024-08-17T02:25:22.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:22.996+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f820815c670 con 0x7f82080b0d20 2024-08-17T02:25:23.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f82057fa700 1 -- 172.21.15.100:0/2412408547 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+12445 (secure 0 0 0) 0x7f81f8017090 con 0x7f82080b0d20 2024-08-17T02:25:23.001 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:23.001 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":44,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:25:20.956928+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6833/1584700240":"2024-08-18T02:25:17.935351+0000","172.21.15.100:6832/1584700240":"2024-08-18T02:25:17.935351+0000","172.21.15.100:6835/3037167008":"2024-08-18T02:25:11.722625+0000","172.21.15.100:6834/3037167008":"2024-08-18T02:25:11.722625+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:25:23.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f81e8060860 msgr2=0x7f81e8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f81e8060860 0x7f81e8062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009de0 tx=0x7f81f0019000 comp rx=0 tx=0).stop 2024-08-17T02:25:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b0d20 msgr2=0x7f82080a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b0d20 0x7f82080a6020 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f81f800c790 tx=0x7f81f8007ea0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 shutdown_connections 2024-08-17T02:25:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f81e8060860 0x7f81e8062d10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82080b1800 0x7f8208146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b0d20 0x7f82080a6020 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 --2- 172.21.15.100:0/2412408547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820806e450 0x7f820816d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 >> 172.21.15.100:0/2412408547 conn(0x7f82080678d0 msgr2=0x7f820806a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:23.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 shutdown_connections 2024-08-17T02:25:23.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.000+0000 7f820e298700 1 -- 172.21.15.100:0/2412408547 wait complete. 2024-08-17T02:25:23.015 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-17T02:25:23.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:25:23.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.116+0000 7f1421f89700 1 Processor -- start 2024-08-17T02:25:23.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.116+0000 7f1421f89700 1 -- start start 2024-08-17T02:25:23.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14140e0a80 0x7f141404fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f14140066a0 0x7f1414053090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1414006cc0 con 0x7f141404ffb0 2024-08-17T02:25:23.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1414037640 con 0x7f14140066a0 2024-08-17T02:25:23.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1414006fa0 con 0x7f14140e0a80 2024-08-17T02:25:23.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f141bfff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f141404ffb0 0x7f1414005f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58332/0 (socket says 172.21.15.100:58332) 2024-08-17T02:25:23.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f141bfff700 1 -- 172.21.15.100:0/2971861209 learned_addr learned my addr 172.21.15.100:0/2971861209 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:23.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421788700 1 --2- 172.21.15.100:0/2971861209 >> v2:172.21.15.100:3300/0 conn(0x7f14140066a0 0x7f1414053090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1420f87700 1 --2- 172.21.15.100:0/2971861209 >> v2:172.21.15.123:3300/0 conn(0x7f14140e0a80 0x7f141404fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f141b7fe700 1 -- 172.21.15.100:0/2971861209 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570300045 0 0) 0x7f1414006cc0 con 0x7f141404ffb0 2024-08-17T02:25:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f141b7fe700 1 -- 172.21.15.100:0/2971861209 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f140c003570 con 0x7f141404ffb0 2024-08-17T02:25:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1420f87700 1 -- 172.21.15.100:0/2971861209 >> v2:172.21.15.100:3300/0 conn(0x7f14140066a0 msgr2=0x7f1414053090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1420f87700 1 --2- 172.21.15.100:0/2971861209 >> v2:172.21.15.100:3300/0 conn(0x7f14140066a0 0x7f1414053090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1420f87700 1 -- 172.21.15.100:0/2971861209 >> v1:172.21.15.100:6789/0 conn(0x7f141404ffb0 legacy=0x7f1414005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1420f87700 1 -- 172.21.15.100:0/2971861209 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414007280 con 0x7f14140e0a80 2024-08-17T02:25:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1420f87700 1 --2- 172.21.15.100:0/2971861209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 0x7f141404fa70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f141c065b20 tx=0x7f141c064ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=194a44f0d0bf9c42 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f141b7fe700 1 -- 172.21.15.100:0/2971861209 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c04c190 con 0x7f14140e0a80 2024-08-17T02:25:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f141b7fe700 1 -- 172.21.15.100:0/2971861209 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c064870 con 0x7f14140e0a80 2024-08-17T02:25:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f141b7fe700 1 -- 172.21.15.100:0/2971861209 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c064c30 con 0x7f14140e0a80 2024-08-17T02:25:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 -- 172.21.15.100:0/2971861209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 msgr2=0x7f141404fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 --2- 172.21.15.100:0/2971861209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 0x7f141404fa70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f141c065b20 tx=0x7f141c064ee0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 -- 172.21.15.100:0/2971861209 shutdown_connections 2024-08-17T02:25:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 --2- 172.21.15.100:0/2971861209 >> v2:172.21.15.100:3300/0 conn(0x7f14140066a0 0x7f1414053090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 --2- 172.21.15.100:0/2971861209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 0x7f141404fa70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 -- 172.21.15.100:0/2971861209 >> 172.21.15.100:0/2971861209 conn(0x7f141400a030 msgr2=0x7f141400a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 -- 172.21.15.100:0/2971861209 shutdown_connections 2024-08-17T02:25:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 -- 172.21.15.100:0/2971861209 wait complete. 2024-08-17T02:25:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.120+0000 7f1421f89700 1 Processor -- start 2024-08-17T02:25:23.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 -- start start 2024-08-17T02:25:23.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14140066a0 0x7f14140380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f141404ffb0 0x7f14140e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 0x7f14140d9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14140e9930 con 0x7f14140066a0 2024-08-17T02:25:23.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141404cef0 con 0x7f14140e0a80 2024-08-17T02:25:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140e9370 con 0x7f141404ffb0 2024-08-17T02:25:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14140066a0 0x7f14140380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14140066a0 0x7f14140380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50584/0 (socket says 172.21.15.100:50584) 2024-08-17T02:25:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 -- 172.21.15.100:0/597219752 learned_addr learned my addr 172.21.15.100:0/597219752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421788700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 0x7f14140d9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f141bfff700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f141404ffb0 0x7f14140e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 -- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f141404ffb0 msgr2=0x7f14140e89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f141404ffb0 0x7f14140e89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 -- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 msgr2=0x7f14140d9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 0x7f14140d9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 -- 172.21.15.100:0/597219752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14140e9650 con 0x7f14140066a0 2024-08-17T02:25:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1420f87700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14140066a0 0x7f14140380c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f141c04ccf0 tx=0x7f141c06f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1418ff9700 1 -- 172.21.15.100:0/597219752 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c04c190 con 0x7f14140066a0 2024-08-17T02:25:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1418ff9700 1 -- 172.21.15.100:0/597219752 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c06d070 con 0x7f14140066a0 2024-08-17T02:25:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141404cb80 con 0x7f14140066a0 2024-08-17T02:25:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1418ff9700 1 -- 172.21.15.100:0/597219752 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0706d0 con 0x7f14140066a0 2024-08-17T02:25:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.124+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14140387a0 con 0x7f14140066a0 2024-08-17T02:25:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.128+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14140053b0 con 0x7f14140066a0 2024-08-17T02:25:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.128+0000 7f1418ff9700 1 -- 172.21.15.100:0/597219752 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f141c070af0 con 0x7f14140066a0 2024-08-17T02:25:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.128+0000 7f1418ff9700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1400060860 0x7f1400062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.128+0000 7f1418ff9700 1 -- 172.21.15.100:0/597219752 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f141c0f44b0 con 0x7f14140066a0 2024-08-17T02:25:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.128+0000 7f141bfff700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1400060860 0x7f1400062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.132+0000 7f141bfff700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1400060860 0x7f1400062d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f141404bba0 tx=0x7f1408009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.136+0000 7f1418ff9700 1 -- 172.21.15.100:0/597219752 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f141c0c4ba0 con 0x7f14140066a0 2024-08-17T02:25:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.296+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1414034880 con 0x7f1400060860 2024-08-17T02:25:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.304+0000 7f1418ff9700 1 -- 172.21.15.100:0/597219752 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49321 (secure 0 0 0) 0x7f1414034880 con 0x7f1400060860 2024-08-17T02:25:23.305 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.304+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1400060860 msgr2=0x7f1400062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.304+0000 7f1421f89700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1400060860 0x7f1400062d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f141404bba0 tx=0x7f1408009040 comp rx=0 tx=0).stop 2024-08-17T02:25:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.304+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14140066a0 msgr2=0x7f14140380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.304+0000 7f1421f89700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14140066a0 0x7f14140380c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f141c04ccf0 tx=0x7f141c06f000 comp rx=0 tx=0).stop 2024-08-17T02:25:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.308+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 shutdown_connections 2024-08-17T02:25:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.308+0000 7f1421f89700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1400060860 0x7f1400062d10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.308+0000 7f1421f89700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14140e0a80 0x7f14140d9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.308+0000 7f1421f89700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f141404ffb0 0x7f14140e89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.308+0000 7f1421f89700 1 --2- 172.21.15.100:0/597219752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14140066a0 0x7f14140380c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.308+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 >> 172.21.15.100:0/597219752 conn(0x7f141400a030 msgr2=0x7f141400a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.308+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 shutdown_connections 2024-08-17T02:25:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.308+0000 7f1421f89700 1 -- 172.21.15.100:0/597219752 wait complete. 2024-08-17T02:25:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:25:23.325 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-08-17T02:25:21.300428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2396320,"kb_used_data":4972,"kb_used_omap":13,"kb_used_meta":2391218,"kb_avail":752578400,"statfs":{"total":773094113280,"available":770640281600,"internally_reserved":0,"allocated":5091328,"data_stored":2784479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2448607936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002004"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":181,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-08-17T02:25:15.306599+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:25:15.306599+0000","last_peered":"2024-08-17T02:25:15.306599+0000","last_clean":"2024-08-17T02:25:15.306599+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:25:15.306599+0000","last_undegraded":"2024-08-17T02:25:15.306599+0000","last_fullsized":"2024-08-17T02:25:15.306599+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298416,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073424,"statfs":{"total":96636764160,"available":96331186176,"internally_reserved":0,"allocated":1085440,"data_stored":794217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91600000000000004,"15min":0.89600000000000002},"min":{"1min":0.68899999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.73699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2769999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.61199999999999999}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2989999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.02},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.875,"15min":0.875},"min":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.175,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.96999999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.84999999999999998,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2290000000000001,"5min":1.306,"15min":1.306},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.98799999999999999,"15min":0.99399999999999999},"min":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2749999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.113}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.98599999999999999},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.242,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.73999999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3839999999999999,"5min":1.897,"15min":1.897},"last":1.2090000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.61499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2390000000000001,"5min":1.288,"15min":1.288},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.004,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.24,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.171}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.321,"5min":1.413,"15min":1.413},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.161,"15min":1.167},"min":{"1min":0.92600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.1499999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.115,"15min":1.115},"min":{"1min":0.81799999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4610000000000001,"5min":1.55,"15min":1.55},"last":1.236},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.135,"15min":1.137},"min":{"1min":0.82399999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3140000000000001,"5min":1.657,"15min":1.657},"last":1.26}]}]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297832,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":483328,"data_stored":198218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.621,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.089,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.70699999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.347,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.92700000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1919999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90500000000000003,"15min":0.90800000000000003},"min":{"1min":0.69499999999999995,"5min":0.621,"15min":0.621},"max":{"1min":1.1850000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.502}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89000000000000001,"15min":0.90300000000000002},"min":{"1min":0.70699999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.234,"15min":1.234},"last":0.626},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.79100000000000004}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.79800000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.038},"min":{"1min":0.71199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.252,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.0580000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.93300000000000005,"15min":0.94699999999999995},"min":{"1min":0.66300000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1220000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.123,"15min":1.159},"min":{"1min":0.83699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.123}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.103,"15min":1.137},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.4199999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.1240000000000001,"15min":1.1479999999999999},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.3779999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.95699999999999996}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.212,"15min":1.2509999999999999},"min":{"1min":0.89100000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.405,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2270000000000001,"15min":1.25},"min":{"1min":1.044,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.4970000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0269999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297836,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":491520,"data_stored":203829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.95899999999999996},"min":{"1min":0.75,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1910000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81499999999999995,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.179},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86899999999999999,"15min":0.89800000000000002},"min":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.248,"5min":1.272,"15min":1.272},"last":0.93100000000000005}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90800000000000003,"15min":0.89400000000000002},"min":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.3029999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.98299999999999998},"min":{"1min":0.67300000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.012}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97499999999999998,"15min":0.98699999999999999},"min":{"1min":0.71999999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.258,"5min":1.429,"15min":1.429},"last":1.052},{"interface":"front","average":{"1min":1.028,"5min":1.083,"15min":1.0960000000000001},"min":{"1min":0.749,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.75,"15min":1.75},"last":1.0760000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95099999999999996,"15min":0.97699999999999998},"min":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.224,"5min":1.506,"15min":1.506},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.04,"15min":1.0740000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.415,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.68300000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.407,"5min":1.855,"15min":1.855},"last":1.244},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0740000000000001,"15min":1.085},"min":{"1min":0.76000000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3600000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.73499999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4039999999999999,"5min":1.831,"15min":1.831},"last":1.216},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":1.0609999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.667,"5min":2.036,"15min":2.036},"last":1.1539999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302004,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069836,"statfs":{"total":96636764160,"available":96327512064,"internally_reserved":0,"allocated":499712,"data_stored":205311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.89800000000000002,"15min":0.91700000000000004},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1279999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1739999999999999,"5min":1.252,"15min":1.252},"last":1.1819999999999999}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0800000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.64700000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1619999999999999,"5min":1.357,"15min":1.357},"last":0.56899999999999995}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2410000000000001,"5min":1.516,"15min":1.516},"last":1.4099999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.72399999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2170000000000001,"5min":1.657,"15min":1.657},"last":1.2629999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.69299999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2190000000000001,"5min":1.625,"15min":1.625},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.079,"5min":1.0469999999999999,"15min":1.0449999999999999},"min":{"1min":0.86099999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2989999999999999,"5min":1.444,"15min":1.444},"last":1.4990000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.99199999999999999,"15min":1.0409999999999999},"min":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0820000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.163,"15min":1.173},"min":{"1min":0.95099999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.5229999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.875,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.2969999999999999,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.1220000000000001,"15min":1.1499999999999999},"min":{"1min":0.71899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.286,"5min":1.474,"15min":1.474},"last":1.3420000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.169,"15min":1.2270000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.26,"5min":3.028,"15min":3.028},"last":1.448},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1839999999999999,"15min":1.202},"min":{"1min":0.998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.28,"5min":1.722,"15min":1.722},"last":1.2290000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301996,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069844,"statfs":{"total":96636764160,"available":96327520256,"internally_reserved":0,"allocated":487424,"data_stored":198184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.70999999999999996},"min":{"1min":0.63900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0720000000000001,"5min":1.105,"15min":1.105},"last":0.73799999999999999}]},{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.622,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":1.056,"15min":1.056},"last":0.57299999999999995}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.65800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":1.327,"15min":1.327},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.65300000000000002}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.94599999999999995,"15min":0.94199999999999995},"min":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.056},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.083}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.0469999999999999,"15min":1.0569999999999999},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.2,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.0840000000000001,"15min":1.0960000000000001},"min":{"1min":0.92200000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2270000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.98999999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0600000000000001},"min":{"1min":0.81999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.27,"5min":1.504,"15min":1.504},"last":1.24},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.1679999999999999},"min":{"1min":0.876,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.3320000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.258}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.92700000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.3109999999999999,"5min":1.784,"15min":1.784},"last":1.137},{"interface":"front","average":{"1min":1.125,"5min":1.103,"15min":1.0880000000000001},"min":{"1min":0.97699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3100000000000001,"5min":1.552,"15min":1.552},"last":1.1579999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297836,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":491520,"data_stored":199132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0589999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.58099999999999996}]},{"osd":3,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.216,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.252,"5min":1.28,"15min":1.28},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.018,"15min":1.02},"min":{"1min":0.78700000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.222,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0449999999999999,"15min":1.0389999999999999},"min":{"1min":0.84099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4319999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.96499999999999997}]},{"osd":5,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0609999999999999,"15min":1.0700000000000001},"min":{"1min":0.89000000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.413,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1080000000000001,"15min":1.115},"min":{"1min":0.86599999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4970000000000001,"5min":1.712,"15min":1.712},"last":0.871}]},{"osd":6,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1160000000000001,"15min":1.1419999999999999},"min":{"1min":0.86899999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.55,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.139,"15min":1.143},"min":{"1min":0.79700000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5800000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.351}]},{"osd":7,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.79000000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6180000000000001,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.03},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.208,"15min":1.218},"min":{"1min":1,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.575,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.403}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301988,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069852,"statfs":{"total":96636764160,"available":96327528448,"internally_reserved":0,"allocated":479232,"data_stored":197628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75600000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58099999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92300000000000004,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76700000000000002,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78300000000000003,"15min":0.77100000000000002},"min":{"1min":0.67600000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98899999999999999,"5min":1.421,"15min":1.421},"last":0.97399999999999998}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85999999999999999,"15min":0.85199999999999998},"min":{"1min":0.69099999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.90900000000000003,"15min":0.94199999999999995},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2,"5min":3.323,"15min":3.323},"last":0.495}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.98299999999999998,"15min":1.008},"min":{"1min":0.81799999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.216,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.72299999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2809999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1000000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0700000000000001,"15min":1.0680000000000001},"min":{"1min":0.80800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.345,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0680000000000001,"15min":1.101},"min":{"1min":0.752,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.1479999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.89000000000000001}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1140000000000001,"15min":1.1220000000000001},"min":{"1min":0.89700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4199999999999999,"5min":1.464,"15min":1.464},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1299999999999999,"15min":1.157},"min":{"1min":0.873,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2410000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.94699999999999995}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.149,"15min":1.159},"min":{"1min":0.97799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.359,"5min":1.605,"15min":1.605},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2110000000000001},"min":{"1min":1.0429999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.351,"5min":1.635,"15min":1.635},"last":0.82899999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298412,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073428,"statfs":{"total":96636764160,"available":96331190272,"internally_reserved":0,"allocated":1073152,"data_stored":787960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.51500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0720000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.53200000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.288,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.501}]},{"osd":3,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0549999999999999,"5min":1.129,"15min":1.129},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.83099999999999996,"15min":0.81399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0920000000000001,"5min":1.105,"15min":1.105},"last":0.53700000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93000000000000005,"15min":0.94399999999999995},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.147,"5min":1.165,"15min":1.165},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.079,"15min":1.1240000000000001},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.294,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.877}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1399999999999999,"15min":1.1879999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.387,"5min":1.573,"15min":1.573},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1879999999999999,"15min":1.214},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.419,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1160000000000001,"15min":1.117},"min":{"1min":0.877,"5min":0.872,"15min":0.872},"max":{"1min":1.4770000000000001,"5min":1.655,"15min":1.655},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.71499999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.357,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.751}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.109},"min":{"1min":0.83399999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1899999999999999,"15min":1.224},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.411,"5min":1.633,"15min":1.633},"last":0.92700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:25:23.327 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:25:23.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.432+0000 7f1833341700 1 Processor -- start 2024-08-17T02:25:23.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.432+0000 7f1833341700 1 -- start start 2024-08-17T02:25:23.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c154900 0x7f182c0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f182c0b12d0 0x7f182c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0cbaa0 con 0x7f182c0cb390 2024-08-17T02:25:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c068790 con 0x7f182c0b12d0 2024-08-17T02:25:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c1583f0 con 0x7f182c154900 2024-08-17T02:25:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1831b3e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f182c0b12d0 0x7f182c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1831b3e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f182c0b12d0 0x7f182c0cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:53704/0 (socket says 172.21.15.100:53704) 2024-08-17T02:25:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1831b3e700 1 -- 172.21.15.100:0/2196568718 learned_addr learned my addr 172.21.15.100:0/2196568718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1832b40700 1 --1- 172.21.15.100:0/2196568718 >> v1:172.21.15.123:6789/0 conn(0x7f182c0cb390 0x7f182c06df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:57884/0 (socket says 172.21.15.100:57884) 2024-08-17T02:25:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f183233f700 1 --2- 172.21.15.100:0/2196568718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c154900 0x7f182c0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1831b3e700 1 -- 172.21.15.100:0/2196568718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c154900 msgr2=0x7f182c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1831b3e700 1 --2- 172.21.15.100:0/2196568718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c154900 0x7f182c0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1831b3e700 1 -- 172.21.15.100:0/2196568718 >> v1:172.21.15.123:6789/0 conn(0x7f182c0cb390 legacy=0x7f182c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1831b3e700 1 -- 172.21.15.100:0/2196568718 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0b1700 con 0x7f182c0b12d0 2024-08-17T02:25:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1831b3e700 1 --2- 172.21.15.100:0/2196568718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b12d0 0x7f182c0cae50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f18200099b0 tx=0x7f1820009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3033135c11ac6155 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f183133d700 1 -- 172.21.15.100:0/2196568718 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820009ea0 con 0x7f182c0b12d0 2024-08-17T02:25:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f183133d700 1 -- 172.21.15.100:0/2196568718 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182000b860 con 0x7f182c0b12d0 2024-08-17T02:25:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f183133d700 1 -- 172.21.15.100:0/2196568718 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182000bc00 con 0x7f182c0b12d0 2024-08-17T02:25:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 -- 172.21.15.100:0/2196568718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b12d0 msgr2=0x7f182c0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 --2- 172.21.15.100:0/2196568718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b12d0 0x7f182c0cae50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f18200099b0 tx=0x7f1820009250 comp rx=0 tx=0).stop 2024-08-17T02:25:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 -- 172.21.15.100:0/2196568718 shutdown_connections 2024-08-17T02:25:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 --2- 172.21.15.100:0/2196568718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b12d0 0x7f182c0cae50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 --2- 172.21.15.100:0/2196568718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c154900 0x7f182c0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 -- 172.21.15.100:0/2196568718 >> 172.21.15.100:0/2196568718 conn(0x7f182c0678d0 msgr2=0x7f182c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 -- 172.21.15.100:0/2196568718 shutdown_connections 2024-08-17T02:25:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.436+0000 7f1833341700 1 -- 172.21.15.100:0/2196568718 wait complete. 2024-08-17T02:25:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1833341700 1 Processor -- start 2024-08-17T02:25:23.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1833341700 1 -- start start 2024-08-17T02:25:23.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1833341700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182c0b12d0 0x7f182c0b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1833341700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0cb390 0x7f182c14a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1833341700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c154900 0x7f182c155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1833341700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0c1e40 con 0x7f182c0b12d0 2024-08-17T02:25:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1833341700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0af930 con 0x7f182c154900 2024-08-17T02:25:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1833341700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0afdc0 con 0x7f182c0cb390 2024-08-17T02:25:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f183233f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182c0b12d0 0x7f182c0b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f183233f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182c0b12d0 0x7f182c0b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50600/0 (socket says 172.21.15.100:50600) 2024-08-17T02:25:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f183233f700 1 -- 172.21.15.100:0/3384998031 learned_addr learned my addr 172.21.15.100:0/3384998031 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.440+0000 7f1832b40700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c154900 0x7f182c155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f1831b3e700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0cb390 0x7f182c14a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f183233f700 1 -- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0cb390 msgr2=0x7f182c14a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f183233f700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0cb390 0x7f182c14a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f183233f700 1 -- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c154900 msgr2=0x7f182c155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f183233f700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c154900 0x7f182c155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f183233f700 1 -- 172.21.15.100:0/3384998031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18200094f0 con 0x7f182c0b12d0 2024-08-17T02:25:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f183233f700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182c0b12d0 0x7f182c0b0590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f181c00ec80 tx=0x7f181c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f182a7fc700 1 -- 172.21.15.100:0/3384998031 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00c850 con 0x7f182c0b12d0 2024-08-17T02:25:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f182a7fc700 1 -- 172.21.15.100:0/3384998031 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00c9b0 con 0x7f182c0b12d0 2024-08-17T02:25:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c0aed20 con 0x7f182c0b12d0 2024-08-17T02:25:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f182a7fc700 1 -- 172.21.15.100:0/3384998031 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00cd50 con 0x7f182c0b12d0 2024-08-17T02:25:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c0af2e0 con 0x7f182c0b12d0 2024-08-17T02:25:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.444+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c0643a0 con 0x7f182c0b12d0 2024-08-17T02:25:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.448+0000 7f182a7fc700 1 -- 172.21.15.100:0/3384998031 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f181c013680 con 0x7f182c0b12d0 2024-08-17T02:25:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.448+0000 7f182a7fc700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1818060860 0x7f1818062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.448+0000 7f182a7fc700 1 -- 172.21.15.100:0/3384998031 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f181c07cb40 con 0x7f182c0b12d0 2024-08-17T02:25:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.448+0000 7f1831b3e700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1818060860 0x7f1818062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.448+0000 7f1831b3e700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1818060860 0x7f1818062d10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f18200099b0 tx=0x7f1820017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.456+0000 7f182a7fc700 1 -- 172.21.15.100:0/3384998031 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f181c04d230 con 0x7f182c0b12d0 2024-08-17T02:25:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.612+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f182c06f5f0 con 0x7f1818060860 2024-08-17T02:25:23.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f182a7fc700 1 -- 172.21.15.100:0/3384998031 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49244 (secure 0 0 0) 0x7f182c06f5f0 con 0x7f1818060860 2024-08-17T02:25:23.624 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1818060860 msgr2=0x7f1818062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1818060860 0x7f1818062d10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f18200099b0 tx=0x7f1820017000 comp rx=0 tx=0).stop 2024-08-17T02:25:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182c0b12d0 msgr2=0x7f182c0b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182c0b12d0 0x7f182c0b0590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f181c00ec80 tx=0x7f181c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 shutdown_connections 2024-08-17T02:25:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1818060860 0x7f1818062d10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c154900 0x7f182c155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0cb390 0x7f182c14a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 --2- 172.21.15.100:0/3384998031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182c0b12d0 0x7f182c0b0590 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.620+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 >> 172.21.15.100:0/3384998031 conn(0x7f182c0678d0 msgr2=0x7f182c06cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.624+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 shutdown_connections 2024-08-17T02:25:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.624+0000 7f1833341700 1 -- 172.21.15.100:0/3384998031 wait complete. 2024-08-17T02:25:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:25:23.637 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-08-17T02:25:23.304490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2400688,"kb_used_data":5192,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752574032,"statfs":{"total":773094113280,"available":770635808768,"internally_reserved":0,"allocated":5316608,"data_stored":2893680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.005376"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-08-17T02:25:20.157962+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:25:20.157962+0000","last_peered":"2024-08-17T02:25:20.157962+0000","last_clean":"2024-08-17T02:25:20.157962+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:25:20.157962+0000","last_undegraded":"2024-08-17T02:25:20.157962+0000","last_fullsized":"2024-08-17T02:25:20.157962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298448,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073392,"statfs":{"total":96636764160,"available":96331153408,"internally_reserved":0,"allocated":1114112,"data_stored":805898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91600000000000004,"15min":0.89600000000000002},"min":{"1min":0.68899999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.73699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2769999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.80700000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2989999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.875,"15min":0.875},"min":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.175,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.1579999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.84999999999999998,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2290000000000001,"5min":1.306,"15min":1.306},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.98799999999999999,"15min":0.99399999999999999},"min":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2749999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.254}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.98599999999999999},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.242,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.73999999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3839999999999999,"5min":1.897,"15min":1.897},"last":1.115}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.61499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2390000000000001,"5min":1.288,"15min":1.288},"last":1.3400000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.004,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.24,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.206}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.321,"5min":1.413,"15min":1.413},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.161,"15min":1.167},"min":{"1min":0.92600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.4119999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.115,"15min":1.115},"min":{"1min":0.81799999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4610000000000001,"5min":1.55,"15min":1.55},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.135,"15min":1.137},"min":{"1min":0.82399999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3140000000000001,"5min":1.657,"15min":1.657},"last":1.048}]}]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.621,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.089,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.70699999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.347,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1919999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90500000000000003,"15min":0.90800000000000003},"min":{"1min":0.69499999999999995,"5min":0.621,"15min":0.621},"max":{"1min":1.1850000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89000000000000001,"15min":0.90300000000000002},"min":{"1min":0.70699999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.234,"15min":1.234},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.79800000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.038},"min":{"1min":0.71199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.252,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.0680000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.93300000000000005,"15min":0.94699999999999995},"min":{"1min":0.66300000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1220000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.123,"15min":1.159},"min":{"1min":0.83699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.1100000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.103,"15min":1.137},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.4199999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.042},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.1240000000000001,"15min":1.1479999999999999},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.3779999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.087}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.212,"15min":1.2509999999999999},"min":{"1min":0.89100000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.405,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2270000000000001,"15min":1.25},"min":{"1min":1.044,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.4970000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.133}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297864,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.95899999999999996},"min":{"1min":0.75,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1910000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81499999999999995,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86899999999999999,"15min":0.89800000000000002},"min":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.248,"5min":1.272,"15min":1.272},"last":0.63100000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90800000000000003,"15min":0.89400000000000002},"min":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.98299999999999998},"min":{"1min":0.67300000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.0649999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97499999999999998,"15min":0.98699999999999999},"min":{"1min":0.71999999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.258,"5min":1.429,"15min":1.429},"last":1.048},{"interface":"front","average":{"1min":1.028,"5min":1.083,"15min":1.0960000000000001},"min":{"1min":0.749,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.75,"15min":1.75},"last":0.86899999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95099999999999996,"15min":0.97699999999999998},"min":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.224,"5min":1.506,"15min":1.506},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.04,"15min":1.0740000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.415,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.68300000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.407,"5min":1.855,"15min":1.855},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0740000000000001,"15min":1.085},"min":{"1min":0.76000000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3600000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.88800000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.73499999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4039999999999999,"5min":1.831,"15min":1.831},"last":1.01},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":1.0609999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.667,"5min":2.036,"15min":2.036},"last":1.0960000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302028,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.89800000000000002,"15min":0.91700000000000004},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1279999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1739999999999999,"5min":1.252,"15min":1.252},"last":0.69099999999999995}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0800000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.64700000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1619999999999999,"5min":1.357,"15min":1.357},"last":1.0049999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2410000000000001,"5min":1.516,"15min":1.516},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.72399999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2170000000000001,"5min":1.657,"15min":1.657},"last":0.83399999999999996}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.69299999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2190000000000001,"5min":1.625,"15min":1.625},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.0469999999999999,"15min":1.0449999999999999},"min":{"1min":0.86099999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2989999999999999,"5min":1.444,"15min":1.444},"last":0.97299999999999998}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.99199999999999999,"15min":1.0409999999999999},"min":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0820000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.163,"15min":1.173},"min":{"1min":0.95099999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.095}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.875,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.2969999999999999,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.1220000000000001,"15min":1.1499999999999999},"min":{"1min":0.71899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.286,"5min":1.474,"15min":1.474},"last":0.86699999999999999}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.169,"15min":1.2270000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.26,"5min":3.028,"15min":3.028},"last":1.071},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1839999999999999,"15min":1.202},"min":{"1min":0.998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.28,"5min":1.722,"15min":1.722},"last":1.2}]}]},{"osd":3,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302024,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.083},{"interface":"front","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.70999999999999996},"min":{"1min":0.63900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0720000000000001,"5min":1.105,"15min":1.105},"last":0.96799999999999997}]},{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.622,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":1.056,"15min":1.056},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.65800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":1.327,"15min":1.327},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.94599999999999995,"15min":0.94199999999999995},"min":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.056},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.1899999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.0469999999999999,"15min":1.0569999999999999},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.2,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.105},{"interface":"front","average":{"1min":1.054,"5min":1.0840000000000001,"15min":1.0960000000000001},"min":{"1min":0.92200000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2270000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.407}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0600000000000001},"min":{"1min":0.81999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.27,"5min":1.504,"15min":1.504},"last":1.236},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.1679999999999999},"min":{"1min":0.876,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.3320000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.375}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.92700000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.3109999999999999,"5min":1.784,"15min":1.784},"last":1.552},{"interface":"front","average":{"1min":1.125,"5min":1.103,"15min":1.0880000000000001},"min":{"1min":0.97699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3100000000000001,"5min":1.552,"15min":1.552},"last":1.331}]}]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297864,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0589999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.216,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.252,"5min":1.28,"15min":1.28},"last":0.77900000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.018,"15min":1.02},"min":{"1min":0.78700000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.222,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0449999999999999,"15min":1.0389999999999999},"min":{"1min":0.84099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4319999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.1000000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0609999999999999,"15min":1.0700000000000001},"min":{"1min":0.89000000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.413,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.123},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1080000000000001,"15min":1.115},"min":{"1min":0.86599999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4970000000000001,"5min":1.712,"15min":1.712},"last":1.0209999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1160000000000001,"15min":1.1419999999999999},"min":{"1min":0.86899999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.55,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.155},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.139,"15min":1.143},"min":{"1min":0.79700000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5800000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.331}]},{"osd":7,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.79000000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6180000000000001,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.208,"15min":1.218},"min":{"1min":1,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.575,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.224}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75600000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58099999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92300000000000004,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76700000000000002,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78300000000000003,"15min":0.77100000000000002},"min":{"1min":0.67600000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98899999999999999,"5min":1.421,"15min":1.421},"last":1.1519999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85999999999999999,"15min":0.85199999999999998},"min":{"1min":0.69099999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.90900000000000003,"15min":0.94199999999999995},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2,"5min":3.323,"15min":3.323},"last":0.51500000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.98299999999999998,"15min":1.008},"min":{"1min":0.81799999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.216,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.204},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.72299999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2809999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.518}]},{"osd":5,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0700000000000001,"15min":1.0680000000000001},"min":{"1min":0.80800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.345,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.4730000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0680000000000001,"15min":1.101},"min":{"1min":0.752,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.1479999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0669999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1140000000000001,"15min":1.1220000000000001},"min":{"1min":0.89700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4199999999999999,"5min":1.464,"15min":1.464},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1299999999999999,"15min":1.157},"min":{"1min":0.873,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2410000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.2689999999999999}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.149,"15min":1.159},"min":{"1min":0.97799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.359,"5min":1.605,"15min":1.605},"last":1.321},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2110000000000001},"min":{"1min":1.0429999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.351,"5min":1.635,"15min":1.635},"last":1.615}]}]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298444,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073396,"statfs":{"total":96636764160,"available":96331157504,"internally_reserved":0,"allocated":1114112,"data_stored":805898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.51500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0720000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.747},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.53200000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.288,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.2749999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0549999999999999,"5min":1.129,"15min":1.129},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.83099999999999996,"15min":0.81399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0920000000000001,"5min":1.105,"15min":1.105},"last":0.85499999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93000000000000005,"15min":0.94399999999999995},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.147,"5min":1.165,"15min":1.165},"last":1.3720000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.079,"15min":1.1240000000000001},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.294,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1399999999999999,"15min":1.1879999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.387,"5min":1.573,"15min":1.573},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1879999999999999,"15min":1.214},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.419,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.24}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1160000000000001,"15min":1.117},"min":{"1min":0.877,"5min":0.872,"15min":0.872},"max":{"1min":1.4770000000000001,"5min":1.655,"15min":1.655},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.71499999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.357,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.04}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.109},"min":{"1min":0.83399999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.181},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1899999999999999,"15min":1.224},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.411,"5min":1.633,"15min":1.633},"last":1.1519999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:25:23.638 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-17T02:25:23.639 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:25:23.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feceeeab700 1 Processor -- start 2024-08-17T02:25:23.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feceeeab700 1 -- start start 2024-08-17T02:25:23.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feceeeab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8155e50 0x7fece80a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feceeeab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fece806f2f0 0x7fece80aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feceeeab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece80afc10 con 0x7fece8156220 2024-08-17T02:25:23.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feceeeab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece80b10b0 con 0x7fece806f2f0 2024-08-17T02:25:23.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feceeeab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece80aed20 con 0x7fece8155e50 2024-08-17T02:25:23.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feced448700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fece806f2f0 0x7fece80aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7fececc47700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8155e50 0x7fece80a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feced448700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fece806f2f0 0x7fece80aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:53712/0 (socket says 172.21.15.100:53712) 2024-08-17T02:25:23.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7fece7fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fece8156220 0x7fece806ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.100:57894/0 (socket says 172.21.15.100:57894) 2024-08-17T02:25:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.744+0000 7feced448700 1 -- 172.21.15.100:0/4287630376 learned_addr learned my addr 172.21.15.100:0/4287630376 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feced448700 1 -- 172.21.15.100:0/4287630376 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8155e50 msgr2=0x7fece80a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feced448700 1 --2- 172.21.15.100:0/4287630376 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8155e50 0x7fece80a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feced448700 1 -- 172.21.15.100:0/4287630376 >> v1:172.21.15.123:6789/0 conn(0x7fece8156220 legacy=0x7fece806ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feced448700 1 -- 172.21.15.100:0/4287630376 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece80af000 con 0x7fece806f2f0 2024-08-17T02:25:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feced448700 1 --2- 172.21.15.100:0/4287630376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece806f2f0 0x7fece80aa420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fece0009f20 tx=0x7fece0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd3f3f8c11fbed5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7fece77fe700 1 -- 172.21.15.100:0/4287630376 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece000e070 con 0x7fece806f2f0 2024-08-17T02:25:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7fece77fe700 1 -- 172.21.15.100:0/4287630376 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece000b9d0 con 0x7fece806f2f0 2024-08-17T02:25:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7fece77fe700 1 -- 172.21.15.100:0/4287630376 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00092e0 con 0x7fece806f2f0 2024-08-17T02:25:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 -- 172.21.15.100:0/4287630376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece806f2f0 msgr2=0x7fece80aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 --2- 172.21.15.100:0/4287630376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece806f2f0 0x7fece80aa420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fece0009f20 tx=0x7fece0009840 comp rx=0 tx=0).stop 2024-08-17T02:25:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 -- 172.21.15.100:0/4287630376 shutdown_connections 2024-08-17T02:25:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 --2- 172.21.15.100:0/4287630376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece806f2f0 0x7fece80aa420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 --2- 172.21.15.100:0/4287630376 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8155e50 0x7fece80a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 -- 172.21.15.100:0/4287630376 >> 172.21.15.100:0/4287630376 conn(0x7fece8066e00 msgr2=0x7fece8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 -- 172.21.15.100:0/4287630376 shutdown_connections 2024-08-17T02:25:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 -- 172.21.15.100:0/4287630376 wait complete. 2024-08-17T02:25:23.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 Processor -- start 2024-08-17T02:25:23.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.748+0000 7feceeeab700 1 -- start start 2024-08-17T02:25:23.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feceeeab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fece806f2f0 0x7fece806a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feceeeab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8155e50 0x7fece80cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feceeeab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8156220 0x7fece80cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feceeeab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece8167d90 con 0x7fece806f2f0 2024-08-17T02:25:23.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feceeeab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece80cb860 con 0x7fece8155e50 2024-08-17T02:25:23.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feceeeab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece8167110 con 0x7fece8156220 2024-08-17T02:25:23.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fece806f2f0 0x7fece806a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fece806f2f0 0x7fece806a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50616/0 (socket says 172.21.15.100:50616) 2024-08-17T02:25:23.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 -- 172.21.15.100:0/1478048943 learned_addr learned my addr 172.21.15.100:0/1478048943 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:23.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fece7fff700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8155e50 0x7fece80cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feced448700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8156220 0x7fece80cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 -- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8156220 msgr2=0x7fece80cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8156220 0x7fece80cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 -- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8155e50 msgr2=0x7fece80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8155e50 0x7fece80cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 -- 172.21.15.100:0/1478048943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0009ae0 con 0x7fece806f2f0 2024-08-17T02:25:23.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fececc47700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fece806f2f0 0x7fece806a760 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fecd400ecb0 tx=0x7fecd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fece4ff9700 1 -- 172.21.15.100:0/1478048943 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400c850 con 0x7fece806f2f0 2024-08-17T02:25:23.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece81673f0 con 0x7fece806f2f0 2024-08-17T02:25:23.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fece4ff9700 1 -- 172.21.15.100:0/1478048943 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd400c9b0 con 0x7fece806f2f0 2024-08-17T02:25:23.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece81679b0 con 0x7fece806f2f0 2024-08-17T02:25:23.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.752+0000 7fece4ff9700 1 -- 172.21.15.100:0/1478048943 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400cd50 con 0x7fece806f2f0 2024-08-17T02:25:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.756+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece80645c0 con 0x7fece806f2f0 2024-08-17T02:25:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.756+0000 7fece4ff9700 1 -- 172.21.15.100:0/1478048943 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fecd4013680 con 0x7fece806f2f0 2024-08-17T02:25:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.756+0000 7fece4ff9700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feccc060860 0x7feccc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.756+0000 7fece4ff9700 1 -- 172.21.15.100:0/1478048943 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fecd407cb40 con 0x7fece806f2f0 2024-08-17T02:25:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.756+0000 7fece7fff700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feccc060860 0x7feccc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.756+0000 7fece7fff700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feccc060860 0x7feccc062d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fece8063cb0 tx=0x7fecdc007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.760+0000 7fece4ff9700 1 -- 172.21.15.100:0/1478048943 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fecd404d230 con 0x7fece806f2f0 2024-08-17T02:25:23.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.924+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fece80ae670 con 0x7feccc060860 2024-08-17T02:25:23.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.928+0000 7fece4ff9700 1 -- 172.21.15.100:0/1478048943 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49244 (secure 0 0 0) 0x7fece80ae670 con 0x7feccc060860 2024-08-17T02:25:23.931 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:23.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feccc060860 msgr2=0x7feccc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feccc060860 0x7feccc062d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fece8063cb0 tx=0x7fecdc007eb0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fece806f2f0 msgr2=0x7fece806a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:23.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fece806f2f0 0x7fece806a760 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fecd400ecb0 tx=0x7fecd40091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 shutdown_connections 2024-08-17T02:25:23.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7feccc060860 0x7feccc062d10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fece8156220 0x7fece80cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8155e50 0x7fece80cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 --2- 172.21.15.100:0/1478048943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fece806f2f0 0x7fece806a760 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:23.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 >> 172.21.15.100:0/1478048943 conn(0x7fece8066e00 msgr2=0x7fece8151910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:23.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 shutdown_connections 2024-08-17T02:25:23.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:23.932+0000 7feceeeab700 1 -- 172.21.15.100:0/1478048943 wait complete. 2024-08-17T02:25:23.935 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:25:23.945 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-08-17T02:25:23.304490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2400688,"kb_used_data":5192,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752574032,"statfs":{"total":773094113280,"available":770635808768,"internally_reserved":0,"allocated":5316608,"data_stored":2893680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.005376"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-08-17T02:25:20.157962+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:25:20.157962+0000","last_peered":"2024-08-17T02:25:20.157962+0000","last_clean":"2024-08-17T02:25:20.157962+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:25:20.157962+0000","last_undegraded":"2024-08-17T02:25:20.157962+0000","last_fullsized":"2024-08-17T02:25:20.157962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298448,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073392,"statfs":{"total":96636764160,"available":96331153408,"internally_reserved":0,"allocated":1114112,"data_stored":805898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91600000000000004,"15min":0.89600000000000002},"min":{"1min":0.68899999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.73699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2769999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.80700000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2989999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.875,"15min":0.875},"min":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.175,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.1579999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.84999999999999998,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2290000000000001,"5min":1.306,"15min":1.306},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.98799999999999999,"15min":0.99399999999999999},"min":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2749999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.254}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.98599999999999999},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.242,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.73999999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3839999999999999,"5min":1.897,"15min":1.897},"last":1.115}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.61499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2390000000000001,"5min":1.288,"15min":1.288},"last":1.3400000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.004,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.24,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.206}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.321,"5min":1.413,"15min":1.413},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.161,"15min":1.167},"min":{"1min":0.92600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.4119999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.115,"15min":1.115},"min":{"1min":0.81799999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4610000000000001,"5min":1.55,"15min":1.55},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.135,"15min":1.137},"min":{"1min":0.82399999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3140000000000001,"5min":1.657,"15min":1.657},"last":1.048}]}]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.621,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.089,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.70699999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.347,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1919999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90500000000000003,"15min":0.90800000000000003},"min":{"1min":0.69499999999999995,"5min":0.621,"15min":0.621},"max":{"1min":1.1850000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89000000000000001,"15min":0.90300000000000002},"min":{"1min":0.70699999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.234,"15min":1.234},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.79800000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.038},"min":{"1min":0.71199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.252,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.0680000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.93300000000000005,"15min":0.94699999999999995},"min":{"1min":0.66300000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1220000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.123,"15min":1.159},"min":{"1min":0.83699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.1100000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.103,"15min":1.137},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.4199999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.042},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.1240000000000001,"15min":1.1479999999999999},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.3779999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.087}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.212,"15min":1.2509999999999999},"min":{"1min":0.89100000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.405,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2270000000000001,"15min":1.25},"min":{"1min":1.044,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.4970000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.133}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297864,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.95899999999999996},"min":{"1min":0.75,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1910000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81499999999999995,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86899999999999999,"15min":0.89800000000000002},"min":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.248,"5min":1.272,"15min":1.272},"last":0.63100000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90800000000000003,"15min":0.89400000000000002},"min":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.98299999999999998},"min":{"1min":0.67300000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.0649999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97499999999999998,"15min":0.98699999999999999},"min":{"1min":0.71999999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.258,"5min":1.429,"15min":1.429},"last":1.048},{"interface":"front","average":{"1min":1.028,"5min":1.083,"15min":1.0960000000000001},"min":{"1min":0.749,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.75,"15min":1.75},"last":0.86899999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95099999999999996,"15min":0.97699999999999998},"min":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.224,"5min":1.506,"15min":1.506},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.04,"15min":1.0740000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.415,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.68300000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.407,"5min":1.855,"15min":1.855},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0740000000000001,"15min":1.085},"min":{"1min":0.76000000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3600000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.88800000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.73499999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4039999999999999,"5min":1.831,"15min":1.831},"last":1.01},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":1.0609999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.667,"5min":2.036,"15min":2.036},"last":1.0960000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302028,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.89800000000000002,"15min":0.91700000000000004},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1279999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1739999999999999,"5min":1.252,"15min":1.252},"last":0.69099999999999995}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0800000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.64700000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1619999999999999,"5min":1.357,"15min":1.357},"last":1.0049999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2410000000000001,"5min":1.516,"15min":1.516},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.72399999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2170000000000001,"5min":1.657,"15min":1.657},"last":0.83399999999999996}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.69299999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2190000000000001,"5min":1.625,"15min":1.625},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.0469999999999999,"15min":1.0449999999999999},"min":{"1min":0.86099999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2989999999999999,"5min":1.444,"15min":1.444},"last":0.97299999999999998}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.99199999999999999,"15min":1.0409999999999999},"min":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0820000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.163,"15min":1.173},"min":{"1min":0.95099999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.095}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.875,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.2969999999999999,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.1220000000000001,"15min":1.1499999999999999},"min":{"1min":0.71899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.286,"5min":1.474,"15min":1.474},"last":0.86699999999999999}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.169,"15min":1.2270000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.26,"5min":3.028,"15min":3.028},"last":1.071},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1839999999999999,"15min":1.202},"min":{"1min":0.998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.28,"5min":1.722,"15min":1.722},"last":1.2}]}]},{"osd":3,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302024,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.083},{"interface":"front","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.70999999999999996},"min":{"1min":0.63900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0720000000000001,"5min":1.105,"15min":1.105},"last":0.96799999999999997}]},{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.622,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":1.056,"15min":1.056},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.65800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":1.327,"15min":1.327},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.94599999999999995,"15min":0.94199999999999995},"min":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.056},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.1899999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.0469999999999999,"15min":1.0569999999999999},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.2,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.105},{"interface":"front","average":{"1min":1.054,"5min":1.0840000000000001,"15min":1.0960000000000001},"min":{"1min":0.92200000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2270000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.407}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0600000000000001},"min":{"1min":0.81999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.27,"5min":1.504,"15min":1.504},"last":1.236},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.1679999999999999},"min":{"1min":0.876,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.3320000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.375}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.92700000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.3109999999999999,"5min":1.784,"15min":1.784},"last":1.552},{"interface":"front","average":{"1min":1.125,"5min":1.103,"15min":1.0880000000000001},"min":{"1min":0.97699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3100000000000001,"5min":1.552,"15min":1.552},"last":1.331}]}]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297864,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0589999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.216,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.252,"5min":1.28,"15min":1.28},"last":0.77900000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.018,"15min":1.02},"min":{"1min":0.78700000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.222,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0449999999999999,"15min":1.0389999999999999},"min":{"1min":0.84099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4319999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.1000000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0609999999999999,"15min":1.0700000000000001},"min":{"1min":0.89000000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.413,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.123},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1080000000000001,"15min":1.115},"min":{"1min":0.86599999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4970000000000001,"5min":1.712,"15min":1.712},"last":1.0209999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1160000000000001,"15min":1.1419999999999999},"min":{"1min":0.86899999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.55,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.155},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.139,"15min":1.143},"min":{"1min":0.79700000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5800000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.331}]},{"osd":7,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.79000000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6180000000000001,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.208,"15min":1.218},"min":{"1min":1,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.575,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.224}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75600000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58099999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92300000000000004,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76700000000000002,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78300000000000003,"15min":0.77100000000000002},"min":{"1min":0.67600000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98899999999999999,"5min":1.421,"15min":1.421},"last":1.1519999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85999999999999999,"15min":0.85199999999999998},"min":{"1min":0.69099999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.90900000000000003,"15min":0.94199999999999995},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2,"5min":3.323,"15min":3.323},"last":0.51500000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.98299999999999998,"15min":1.008},"min":{"1min":0.81799999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.216,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.204},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.72299999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2809999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.518}]},{"osd":5,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0700000000000001,"15min":1.0680000000000001},"min":{"1min":0.80800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.345,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.4730000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0680000000000001,"15min":1.101},"min":{"1min":0.752,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.1479999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0669999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1140000000000001,"15min":1.1220000000000001},"min":{"1min":0.89700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4199999999999999,"5min":1.464,"15min":1.464},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1299999999999999,"15min":1.157},"min":{"1min":0.873,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2410000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.2689999999999999}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.149,"15min":1.159},"min":{"1min":0.97799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.359,"5min":1.605,"15min":1.605},"last":1.321},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2110000000000001},"min":{"1min":1.0429999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.351,"5min":1.635,"15min":1.635},"last":1.615}]}]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298444,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073396,"statfs":{"total":96636764160,"available":96331157504,"internally_reserved":0,"allocated":1114112,"data_stored":805898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.51500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0720000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.747},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.53200000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.288,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.2749999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0549999999999999,"5min":1.129,"15min":1.129},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.83099999999999996,"15min":0.81399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0920000000000001,"5min":1.105,"15min":1.105},"last":0.85499999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93000000000000005,"15min":0.94399999999999995},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.147,"5min":1.165,"15min":1.165},"last":1.3720000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.079,"15min":1.1240000000000001},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.294,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1399999999999999,"15min":1.1879999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.387,"5min":1.573,"15min":1.573},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1879999999999999,"15min":1.214},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.419,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.24}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1160000000000001,"15min":1.117},"min":{"1min":0.877,"5min":0.872,"15min":0.872},"max":{"1min":1.4770000000000001,"5min":1.655,"15min":1.655},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.71499999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.357,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.04}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.109},"min":{"1min":0.83399999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.181},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1899999999999999,"15min":1.224},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.411,"5min":1.633,"15min":1.633},"last":1.1519999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:25:23.946 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:25:24.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.044+0000 7fe1a65d2700 1 Processor -- start 2024-08-17T02:25:24.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.044+0000 7fe1a65d2700 1 -- start start 2024-08-17T02:25:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0159040 0x7fe1a00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1a00d0010 0x7fe1a00d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a00cf190 con 0x7fe1a015cea0 2024-08-17T02:25:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a00d09a0 con 0x7fe1a00d0010 2024-08-17T02:25:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00ce5e0 con 0x7fe1a0159040 2024-08-17T02:25:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a4dcf700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1a00d0010 0x7fe1a00d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a4dcf700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1a00d0010 0x7fe1a00d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50628/0 (socket says 172.21.15.100:50628) 2024-08-17T02:25:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a4dcf700 1 -- 172.21.15.100:0/1420312689 learned_addr learned my addr 172.21.15.100:0/1420312689 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a5dd1700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe1a015cea0 0x7fe1a015d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58334/0 (socket says 172.21.15.100:58334) 2024-08-17T02:25:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a55d0700 1 --2- 172.21.15.100:0/1420312689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0159040 0x7fe1a00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a4dcf700 1 -- 172.21.15.100:0/1420312689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0159040 msgr2=0x7fe1a00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a4dcf700 1 --2- 172.21.15.100:0/1420312689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0159040 0x7fe1a00cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a4dcf700 1 -- 172.21.15.100:0/1420312689 >> v1:172.21.15.100:6789/0 conn(0x7fe1a015cea0 legacy=0x7fe1a015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a4dcf700 1 -- 172.21.15.100:0/1420312689 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00ce860 con 0x7fe1a00d0010 2024-08-17T02:25:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a4dcf700 1 --2- 172.21.15.100:0/1420312689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a00d0010 0x7fe1a00d0460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0099b0 tx=0x7fe19c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c593588382e5fa1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe197fff700 1 -- 172.21.15.100:0/1420312689 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c00b860 con 0x7fe1a00d0010 2024-08-17T02:25:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe197fff700 1 -- 172.21.15.100:0/1420312689 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19c00b9c0 con 0x7fe1a00d0010 2024-08-17T02:25:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe197fff700 1 -- 172.21.15.100:0/1420312689 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c00bd60 con 0x7fe1a00d0010 2024-08-17T02:25:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 -- 172.21.15.100:0/1420312689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a00d0010 msgr2=0x7fe1a00d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/1420312689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a00d0010 0x7fe1a00d0460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0099b0 tx=0x7fe19c009250 comp rx=0 tx=0).stop 2024-08-17T02:25:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 -- 172.21.15.100:0/1420312689 shutdown_connections 2024-08-17T02:25:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/1420312689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a00d0010 0x7fe1a00d0460 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/1420312689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0159040 0x7fe1a00cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 -- 172.21.15.100:0/1420312689 >> 172.21.15.100:0/1420312689 conn(0x7fe1a0068250 msgr2=0x7fe1a0068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 -- 172.21.15.100:0/1420312689 shutdown_connections 2024-08-17T02:25:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.048+0000 7fe1a65d2700 1 -- 172.21.15.100:0/1420312689 wait complete. 2024-08-17T02:25:24.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 Processor -- start 2024-08-17T02:25:24.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 -- start start 2024-08-17T02:25:24.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a00d0010 0x7fe1a014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:24.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a0159040 0x7fe1a0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:24.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a015cea0 0x7fe1a01570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:24.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a01502f0 con 0x7fe1a0159040 2024-08-17T02:25:24.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a016a900 con 0x7fe1a00d0010 2024-08-17T02:25:24.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a016a470 con 0x7fe1a015cea0 2024-08-17T02:25:24.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a0159040 0x7fe1a0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:24.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a0159040 0x7fe1a0156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50630/0 (socket says 172.21.15.100:50630) 2024-08-17T02:25:24.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 -- 172.21.15.100:0/979053028 learned_addr learned my addr 172.21.15.100:0/979053028 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:24.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a5dd1700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a015cea0 0x7fe1a01570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:24.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a55d0700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a00d0010 0x7fe1a014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:24.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 -- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a015cea0 msgr2=0x7fe1a01570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:24.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a015cea0 0x7fe1a01570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 -- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a00d0010 msgr2=0x7fe1a014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:24.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a00d0010 0x7fe1a014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 -- 172.21.15.100:0/979053028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c0094f0 con 0x7fe1a0159040 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a5dd1700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a015cea0 0x7fe1a01570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a55d0700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a00d0010 0x7fe1a014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a4dcf700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a0159040 0x7fe1a0156ba0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe19c006fd0 tx=0x7fe19c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1957fa700 1 -- 172.21.15.100:0/979053028 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c01a420 con 0x7fe1a0159040 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1957fa700 1 -- 172.21.15.100:0/979053028 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19c01a5a0 con 0x7fe1a0159040 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a01698e0 con 0x7fe1a0159040 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1957fa700 1 -- 172.21.15.100:0/979053028 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c01a960 con 0x7fe1a0159040 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.052+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a0169e20 con 0x7fe1a0159040 2024-08-17T02:25:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.056+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a00b5e30 con 0x7fe1a0159040 2024-08-17T02:25:24.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.056+0000 7fe1957fa700 1 -- 172.21.15.100:0/979053028 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe19c01ad10 con 0x7fe1a0159040 2024-08-17T02:25:24.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.056+0000 7fe1957fa700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe184060860 0x7fe184062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.056+0000 7fe1957fa700 1 -- 172.21.15.100:0/979053028 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe19c07e300 con 0x7fe1a0159040 2024-08-17T02:25:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.056+0000 7fe1a55d0700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe184060860 0x7fe184062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.056+0000 7fe1a55d0700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe184060860 0x7fe184062d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe190006fd0 tx=0x7fe190008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.064+0000 7fe1957fa700 1 -- 172.21.15.100:0/979053028 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe1a00b5e30 con 0x7fe1a0159040 2024-08-17T02:25:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.228+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1a00b5e30 con 0x7fe1a0159040 2024-08-17T02:25:24.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.228+0000 7fe1957fa700 1 -- 172.21.15.100:0/979053028 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+12445 (secure 0 0 0) 0x7fe1a00b5e30 con 0x7fe1a0159040 2024-08-17T02:25:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:24.232 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":44,"fsid":"147b3f96-a106-4648-b442-f3eda3a290c0","created":"2024-08-17T02:21:46.993497+0000","modified":"2024-08-17T02:25:20.956928+0000","last_up_change":"2024-08-17T02:21:57.907276+0000","last_in_change":"2024-08-17T02:21:49.996480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:21:59.284341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"1efac916-de27-46ba-9482-a634bf91666d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":24271},{"type":"v1","addr":"172.21.15.100:6801","nonce":24271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":24271},{"type":"v1","addr":"172.21.15.100:6803","nonce":24271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":24271},{"type":"v1","addr":"172.21.15.100:6807","nonce":24271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":24271},{"type":"v1","addr":"172.21.15.100:6805","nonce":24271}]},"public_addr":"172.21.15.100:6801/24271","cluster_addr":"172.21.15.100:6803/24271","heartbeat_back_addr":"172.21.15.100:6807/24271","heartbeat_front_addr":"172.21.15.100:6805/24271","state":["exists","up"]},{"osd":1,"uuid":"f8ea8c11-8a12-4112-9405-41fb3b5f66e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":24273},{"type":"v1","addr":"172.21.15.100:6825","nonce":24273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":24273},{"type":"v1","addr":"172.21.15.100:6827","nonce":24273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":24273},{"type":"v1","addr":"172.21.15.100:6831","nonce":24273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":24273},{"type":"v1","addr":"172.21.15.100:6829","nonce":24273}]},"public_addr":"172.21.15.100:6825/24273","cluster_addr":"172.21.15.100:6827/24273","heartbeat_back_addr":"172.21.15.100:6831/24273","heartbeat_front_addr":"172.21.15.100:6829/24273","state":["exists","up"]},{"osd":2,"uuid":"a9cfbe5e-93a1-4a49-bc34-b4fd249d41b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":24272},{"type":"v1","addr":"172.21.15.100:6810","nonce":24272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":24272},{"type":"v1","addr":"172.21.15.100:6814","nonce":24272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6819","nonce":24272},{"type":"v1","addr":"172.21.15.100:6820","nonce":24272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":24272},{"type":"v1","addr":"172.21.15.100:6817","nonce":24272}]},"public_addr":"172.21.15.100:6810/24272","cluster_addr":"172.21.15.100:6814/24272","heartbeat_back_addr":"172.21.15.100:6820/24272","heartbeat_front_addr":"172.21.15.100:6817/24272","state":["exists","up"]},{"osd":3,"uuid":"820181f2-20e4-4352-86bf-b9c176577bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":24274},{"type":"v1","addr":"172.21.15.100:6811","nonce":24274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":24274},{"type":"v1","addr":"172.21.15.100:6816","nonce":24274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":24274},{"type":"v1","addr":"172.21.15.100:6823","nonce":24274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":24274},{"type":"v1","addr":"172.21.15.100:6821","nonce":24274}]},"public_addr":"172.21.15.100:6811/24274","cluster_addr":"172.21.15.100:6816/24274","heartbeat_back_addr":"172.21.15.100:6823/24274","heartbeat_front_addr":"172.21.15.100:6821/24274","state":["exists","up"]},{"osd":4,"uuid":"eb72a4df-7be4-4e0c-a44f-d2b8e7e1f4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":24439},{"type":"v1","addr":"172.21.15.123:6805","nonce":24439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":24439},{"type":"v1","addr":"172.21.15.123:6808","nonce":24439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":24439},{"type":"v1","addr":"172.21.15.123:6814","nonce":24439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":24439},{"type":"v1","addr":"172.21.15.123:6811","nonce":24439}]},"public_addr":"172.21.15.123:6805/24439","cluster_addr":"172.21.15.123:6808/24439","heartbeat_back_addr":"172.21.15.123:6814/24439","heartbeat_front_addr":"172.21.15.123:6811/24439","state":["exists","up"]},{"osd":5,"uuid":"a54b7558-76f4-4ff3-8022-a2dbf14c709d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":24438},{"type":"v1","addr":"172.21.15.123:6801","nonce":24438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":24438},{"type":"v1","addr":"172.21.15.123:6804","nonce":24438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":24438},{"type":"v1","addr":"172.21.15.123:6815","nonce":24438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":24438},{"type":"v1","addr":"172.21.15.123:6809","nonce":24438}]},"public_addr":"172.21.15.123:6801/24438","cluster_addr":"172.21.15.123:6804/24438","heartbeat_back_addr":"172.21.15.123:6815/24438","heartbeat_front_addr":"172.21.15.123:6809/24438","state":["exists","up"]},{"osd":6,"uuid":"ce6e56ba-598d-4fb4-8841-69ec22272908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":24440},{"type":"v1","addr":"172.21.15.123:6817","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":24440},{"type":"v1","addr":"172.21.15.123:6819","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":24440},{"type":"v1","addr":"172.21.15.123:6823","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":24440},{"type":"v1","addr":"172.21.15.123:6821","nonce":24440}]},"public_addr":"172.21.15.123:6817/24440","cluster_addr":"172.21.15.123:6819/24440","heartbeat_back_addr":"172.21.15.123:6823/24440","heartbeat_front_addr":"172.21.15.123:6821/24440","state":["exists","up"]},{"osd":7,"uuid":"220ff9d3-4fba-41b9-8c18-68374b5d22aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":24441},{"type":"v1","addr":"172.21.15.123:6825","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":24441},{"type":"v1","addr":"172.21.15.123:6827","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":24441},{"type":"v1","addr":"172.21.15.123:6831","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":24441},{"type":"v1","addr":"172.21.15.123:6829","nonce":24441}]},"public_addr":"172.21.15.123:6825/24441","cluster_addr":"172.21.15.123:6827/24441","heartbeat_back_addr":"172.21.15.123:6831/24441","heartbeat_front_addr":"172.21.15.123:6829/24441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.162568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.405184+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.008892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.224993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.002338+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.235139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:21:55.168232+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6833/1584700240":"2024-08-18T02:25:17.935351+0000","172.21.15.100:6832/1584700240":"2024-08-18T02:25:17.935351+0000","172.21.15.100:6835/3037167008":"2024-08-18T02:25:11.722625+0000","172.21.15.100:6834/3037167008":"2024-08-18T02:25:11.722625+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:25:24.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe184060860 msgr2=0x7fe184062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:24.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe184060860 0x7fe184062d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe190006fd0 tx=0x7fe190008040 comp rx=0 tx=0).stop 2024-08-17T02:25:24.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a0159040 msgr2=0x7fe1a0156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:24.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a0159040 0x7fe1a0156ba0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe19c006fd0 tx=0x7fe19c017000 comp rx=0 tx=0).stop 2024-08-17T02:25:24.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 shutdown_connections 2024-08-17T02:25:24.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe184060860 0x7fe184062d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a015cea0 0x7fe1a01570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a0159040 0x7fe1a0156ba0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 --2- 172.21.15.100:0/979053028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a00d0010 0x7fe1a014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:24.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 >> 172.21.15.100:0/979053028 conn(0x7fe1a0068250 msgr2=0x7fe1a0157e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:24.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 shutdown_connections 2024-08-17T02:25:24.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:24.232+0000 7fe1a65d2700 1 -- 172.21.15.100:0/979053028 wait complete. 2024-08-17T02:25:25.247 INFO:tasks.ceph:Scrubbing osd.0 2024-08-17T02:25:25.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.368+0000 7f19b1990700 1 Processor -- start 2024-08-17T02:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.368+0000 7f19b1990700 1 -- start start 2024-08-17T02:25:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19ac0cd2b0 0x7f19ac0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0caeb0 0x7f19ac0ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19ac0cb300 0x7f19ac0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac0cc480 con 0x7f19ac0cd2b0 2024-08-17T02:25:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac159720 con 0x7f19ac0cb300 2024-08-17T02:25:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0cb870 con 0x7f19ac0caeb0 2024-08-17T02:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19ac0cd2b0 0x7f19ac0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19ab7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19ac0cb300 0x7f19ac0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19ac0cd2b0 0x7f19ac0cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48002/0 (socket says 172.21.15.100:48002) 2024-08-17T02:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 -- 172.21.15.100:0/581114744 learned_addr learned my addr 172.21.15.100:0/581114744 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aa7fc700 1 --2- 172.21.15.100:0/581114744 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0caeb0 0x7f19ac0ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 -- 172.21.15.100:0/581114744 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0caeb0 msgr2=0x7f19ac0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:25.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 --2- 172.21.15.100:0/581114744 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0caeb0 0x7f19ac0ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 -- 172.21.15.100:0/581114744 >> v2:172.21.15.123:3300/0 conn(0x7f19ac0cb300 msgr2=0x7f19ac0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:25.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 --2- 172.21.15.100:0/581114744 >> v2:172.21.15.123:3300/0 conn(0x7f19ac0cb300 0x7f19ac0ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 -- 172.21.15.100:0/581114744 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac0cbb50 con 0x7f19ac0cd2b0 2024-08-17T02:25:25.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19aaffd700 1 --2- 172.21.15.100:0/581114744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0cd2b0 0x7f19ac0cd680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0cc3f0 tx=0x7f19a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7507cecfd317f86a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19a9ffb700 1 -- 172.21.15.100:0/581114744 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000b640 con 0x7f19ac0cd2b0 2024-08-17T02:25:25.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19a9ffb700 1 -- 172.21.15.100:0/581114744 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a000b7a0 con 0x7f19ac0cd2b0 2024-08-17T02:25:25.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19a9ffb700 1 -- 172.21.15.100:0/581114744 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000bb60 con 0x7f19ac0cd2b0 2024-08-17T02:25:25.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 -- 172.21.15.100:0/581114744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0cd2b0 msgr2=0x7f19ac0cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 --2- 172.21.15.100:0/581114744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0cd2b0 0x7f19ac0cd680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0cc3f0 tx=0x7f19a0009250 comp rx=0 tx=0).stop 2024-08-17T02:25:25.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 -- 172.21.15.100:0/581114744 shutdown_connections 2024-08-17T02:25:25.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 --2- 172.21.15.100:0/581114744 >> v2:172.21.15.123:3300/0 conn(0x7f19ac0cb300 0x7f19ac0ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 --2- 172.21.15.100:0/581114744 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0caeb0 0x7f19ac0ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 --2- 172.21.15.100:0/581114744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0cd2b0 0x7f19ac0cd680 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 -- 172.21.15.100:0/581114744 >> 172.21.15.100:0/581114744 conn(0x7f19ac099550 msgr2=0x7f19ac067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:25.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 -- 172.21.15.100:0/581114744 shutdown_connections 2024-08-17T02:25:25.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.372+0000 7f19b1990700 1 -- 172.21.15.100:0/581114744 wait complete. 2024-08-17T02:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19b1990700 1 Processor -- start 2024-08-17T02:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19b1990700 1 -- start start 2024-08-17T02:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19b1990700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0caeb0 0x7f19ac0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19b1990700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0cb300 0x7f19ac0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19b1990700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac0cd2b0 0x7f19ac0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19b1990700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0b5330 con 0x7f19ac0caeb0 2024-08-17T02:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19b1990700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0b5e00 con 0x7f19ac0cd2b0 2024-08-17T02:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19b1990700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0b5560 con 0x7f19ac0cb300 2024-08-17T02:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0caeb0 0x7f19ac0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0caeb0 0x7f19ac0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48004/0 (socket says 172.21.15.100:48004) 2024-08-17T02:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac0cd2b0 0x7f19ac0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 -- 172.21.15.100:0/1704951141 learned_addr learned my addr 172.21.15.100:0/1704951141 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aa7fc700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0cb300 0x7f19ac0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 -- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0cb300 msgr2=0x7f19ac0b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19ac0cb300 0x7f19ac0b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 -- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac0cd2b0 msgr2=0x7f19ac0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac0cd2b0 0x7f19ac0afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 -- 172.21.15.100:0/1704951141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00094f0 con 0x7f19ac0caeb0 2024-08-17T02:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.376+0000 7f19aaffd700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0caeb0 0x7f19ac0b27e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f19a00094c0 tx=0x7f19a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0000c90 con 0x7f19ac0caeb0 2024-08-17T02:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a000b910 con 0x7f19ac0caeb0 2024-08-17T02:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac0b5840 con 0x7f19ac0caeb0 2024-08-17T02:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0026430 con 0x7f19ac0caeb0 2024-08-17T02:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac0c8410 con 0x7f19ac0caeb0 2024-08-17T02:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19ac0638f0 con 0x7f19ac0caeb0 2024-08-17T02:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f19a0023030 con 0x7f19ac0caeb0 2024-08-17T02:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f1992ffd700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1998060860 0x7f1998062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.380+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f19a007e6c0 con 0x7f19ac0caeb0 2024-08-17T02:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.384+0000 7f19aa7fc700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1998060860 0x7f1998062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.384+0000 7f1992ffd700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f1998066750 0x7f1998068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.384+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 --> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1998069210 con 0x7f1998066750 2024-08-17T02:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.384+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a007ea90 con 0x7f19ac0caeb0 2024-08-17T02:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.384+0000 7f19ab7fe700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f1998066750 0x7f1998068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.384+0000 7f19aa7fc700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1998060860 0x7f1998062d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0ca110 tx=0x7f199c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.384+0000 7f19ab7fe700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f1998066750 0x7f1998068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.384+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 <== osd.0 v2:172.21.15.100:6800/24271 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1998069210 con 0x7f1998066750 2024-08-17T02:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.400+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 --> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f19ac064c40 con 0x7f1998066750 2024-08-17T02:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f1992ffd700 1 -- 172.21.15.100:0/1704951141 <== osd.0 v2:172.21.15.100:6800/24271 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f19ac064c40 con 0x7f1998066750 2024-08-17T02:25:25.404 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:25:25.405 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:25:25.405 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:25:25.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f1998066750 msgr2=0x7f1998068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.100:6800/24271,v1:172.21.15.100:6801/24271] conn(0x7f1998066750 0x7f1998068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1998060860 msgr2=0x7f1998062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1998060860 0x7f1998062d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0ca110 tx=0x7f199c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0caeb0 msgr2=0x7f19ac0b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 --2- 172.21.15.100:0/1704951141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ac0caeb0 0x7f19ac0b27e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f19a00094c0 tx=0x7f19a0017000 comp rx=0 tx=0).stop 2024-08-17T02:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19ab7fe700 1 -- 172.21.15.100:0/1704951141 reap_dead start 2024-08-17T02:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 shutdown_connections 2024-08-17T02:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 >> 172.21.15.100:0/1704951141 conn(0x7f19ac099550 msgr2=0x7f19ac0acec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 shutdown_connections 2024-08-17T02:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.404+0000 7f19b1990700 1 -- 172.21.15.100:0/1704951141 wait complete. 2024-08-17T02:25:25.422 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-08-17T02:25:25.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9f26a700 1 Processor -- start 2024-08-17T02:25:25.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9f26a700 1 -- start start 2024-08-17T02:25:25.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9f26a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2f980ab540 0x7f2f9806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9f26a700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980aba70 con 0x7f2f98155720 2024-08-17T02:25:25.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9f26a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980abd50 con 0x7f2f9806e9d0 2024-08-17T02:25:25.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9f26a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980ac030 con 0x7f2f980ab540 2024-08-17T02:25:25.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9ea69700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2f98155720 0x7f2f980acd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56006/0 (socket says 172.21.15.100:56006) 2024-08-17T02:25:25.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9ea69700 1 -- 172.21.15.100:0/3705194807 learned_addr learned my addr 172.21.15.100:0/3705194807 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:25.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.532+0000 7f2f9da67700 1 --2- 172.21.15.100:0/3705194807 >> v2:172.21.15.123:3300/0 conn(0x7f2f980ab540 0x7f2f9806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9d266700 1 -- 172.21.15.100:0/3705194807 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333032814 0 0) 0x7f2f980aba70 con 0x7f2f98155720 2024-08-17T02:25:25.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9d266700 1 -- 172.21.15.100:0/3705194807 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f84004680 con 0x7f2f98155720 2024-08-17T02:25:25.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9da67700 1 -- 172.21.15.100:0/3705194807 >> v1:172.21.15.123:6789/0 conn(0x7f2f9806e9d0 legacy=0x7f2f98155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9da67700 1 -- 172.21.15.100:0/3705194807 >> v1:172.21.15.100:6789/0 conn(0x7f2f98155720 legacy=0x7f2f980acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9da67700 1 -- 172.21.15.100:0/3705194807 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980ac310 con 0x7f2f980ab540 2024-08-17T02:25:25.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9da67700 1 --2- 172.21.15.100:0/3705194807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 0x7f2f9806eda0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2f880098e0 tx=0x7f2f880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99978a88f1edbf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9d266700 1 -- 172.21.15.100:0/3705194807 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8800b640 con 0x7f2f980ab540 2024-08-17T02:25:25.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9d266700 1 -- 172.21.15.100:0/3705194807 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8800b7a0 con 0x7f2f980ab540 2024-08-17T02:25:25.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9d266700 1 -- 172.21.15.100:0/3705194807 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8800bb20 con 0x7f2f980ab540 2024-08-17T02:25:25.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3705194807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 msgr2=0x7f2f9806eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9f26a700 1 --2- 172.21.15.100:0/3705194807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 0x7f2f9806eda0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2f880098e0 tx=0x7f2f880091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3705194807 shutdown_connections 2024-08-17T02:25:25.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9f26a700 1 --2- 172.21.15.100:0/3705194807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 0x7f2f9806eda0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3705194807 >> 172.21.15.100:0/3705194807 conn(0x7f2f980678d0 msgr2=0x7f2f98067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:25.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3705194807 shutdown_connections 2024-08-17T02:25:25.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.536+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3705194807 wait complete. 2024-08-17T02:25:25.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 Processor -- start 2024-08-17T02:25:25.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 -- start start 2024-08-17T02:25:25.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f9806e9d0 0x7f2f98169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 0x7f2f981564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f98155720 0x7f2f98141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9816a140 con 0x7f2f98155720 2024-08-17T02:25:25.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9816a420 con 0x7f2f980ab540 2024-08-17T02:25:25.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9816a700 con 0x7f2f9806e9d0 2024-08-17T02:25:25.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f98155720 0x7f2f98141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f98155720 0x7f2f98141750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48020/0 (socket says 172.21.15.100:48020) 2024-08-17T02:25:25.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 -- 172.21.15.100:0/3541977281 learned_addr learned my addr 172.21.15.100:0/3541977281 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:25.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9e268700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f9806e9d0 0x7f2f98169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9da67700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 0x7f2f981564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 -- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f9806e9d0 msgr2=0x7f2f98169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f9806e9d0 0x7f2f98169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 -- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 msgr2=0x7f2f981564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 0x7f2f981564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 -- 172.21.15.100:0/3541977281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88009420 con 0x7f2f98155720 2024-08-17T02:25:25.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9ea69700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f98155720 0x7f2f98141750 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001460 tx=0x7f2f90001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f8e7fc700 1 -- 172.21.15.100:0/3541977281 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000a870 con 0x7f2f98155720 2024-08-17T02:25:25.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f8e7fc700 1 -- 172.21.15.100:0/3541977281 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9000a9d0 con 0x7f2f98155720 2024-08-17T02:25:25.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9816a9e0 con 0x7f2f98155720 2024-08-17T02:25:25.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f8e7fc700 1 -- 172.21.15.100:0/3541977281 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000ad50 con 0x7f2f98155720 2024-08-17T02:25:25.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.540+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f98156a20 con 0x7f2f98155720 2024-08-17T02:25:25.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.544+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f980a9b20 con 0x7f2f98155720 2024-08-17T02:25:25.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.544+0000 7f2f8e7fc700 1 -- 172.21.15.100:0/3541977281 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2f90007300 con 0x7f2f98155720 2024-08-17T02:25:25.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.544+0000 7f2f8e7fc700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2f7c060860 0x7f2f7c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.544+0000 7f2f8e7fc700 1 -- 172.21.15.100:0/3541977281 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f2f9007cb90 con 0x7f2f98155720 2024-08-17T02:25:25.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.544+0000 7f2f9e268700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2f7c060860 0x7f2f7c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.544+0000 7f2f9e268700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2f7c060860 0x7f2f7c062d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2f98069f00 tx=0x7f2f94009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.556+0000 7f2f8e7fc700 1 -- 172.21.15.100:0/3541977281 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2f9004d300 con 0x7f2f98155720 2024-08-17T02:25:25.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.716+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f9816a870 con 0x7f2f7c060860 2024-08-17T02:25:25.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.716+0000 7f2f8e7fc700 1 -- 172.21.15.100:0/3541977281 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2f9816a870 con 0x7f2f7c060860 2024-08-17T02:25:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2f7c060860 msgr2=0x7f2f7c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2f7c060860 0x7f2f7c062d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2f98069f00 tx=0x7f2f94009040 comp rx=0 tx=0).stop 2024-08-17T02:25:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f98155720 msgr2=0x7f2f98141750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f98155720 0x7f2f98141750 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001460 tx=0x7f2f90001a40 comp rx=0 tx=0).stop 2024-08-17T02:25:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 shutdown_connections 2024-08-17T02:25:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2f7c060860 0x7f2f7c062d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f98155720 0x7f2f98141750 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f980ab540 0x7f2f981564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 --2- 172.21.15.100:0/3541977281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f9806e9d0 0x7f2f98169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 >> 172.21.15.100:0/3541977281 conn(0x7f2f980678d0 msgr2=0x7f2f980adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 shutdown_connections 2024-08-17T02:25:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.720+0000 7f2f9f26a700 1 -- 172.21.15.100:0/3541977281 wait complete. 2024-08-17T02:25:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 0 to deep-scrub 2024-08-17T02:25:25.736 INFO:tasks.ceph:Scrubbing osd.1 2024-08-17T02:25:25.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:25:25.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878ffff700 1 Processor -- start 2024-08-17T02:25:25.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878ffff700 1 -- start start 2024-08-17T02:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8790155520 0x7f87900cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878ffff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8790159f20 0x7f879006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879006dec0 con 0x7f87901593c0 2024-08-17T02:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878ffff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87900cbab0 con 0x7f8790159f20 2024-08-17T02:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87900ae220 con 0x7f8790155520 2024-08-17T02:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878f7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8790159f20 0x7f879006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878f7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8790159f20 0x7f879006e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48028/0 (socket says 172.21.15.100:48028) 2024-08-17T02:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878f7fe700 1 -- 172.21.15.100:0/2054589019 learned_addr learned my addr 172.21.15.100:0/2054589019 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878effd700 1 --2- 172.21.15.100:0/2054589019 >> v2:172.21.15.123:3300/0 conn(0x7f8790155520 0x7f87900cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878f7fe700 1 -- 172.21.15.100:0/2054589019 >> v2:172.21.15.123:3300/0 conn(0x7f8790155520 msgr2=0x7f87900cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878f7fe700 1 --2- 172.21.15.100:0/2054589019 >> v2:172.21.15.123:3300/0 conn(0x7f8790155520 0x7f87900cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878f7fe700 1 -- 172.21.15.100:0/2054589019 >> v1:172.21.15.123:6789/0 conn(0x7f87901593c0 legacy=0x7f8790159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878f7fe700 1 -- 172.21.15.100:0/2054589019 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87900ae500 con 0x7f8790159f20 2024-08-17T02:25:25.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878f7fe700 1 --2- 172.21.15.100:0/2054589019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 0x7f879006e350 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8790064cb0 tx=0x7f878800a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa5c6ade899d8a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878dffb700 1 -- 172.21.15.100:0/2054589019 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800f070 con 0x7f8790159f20 2024-08-17T02:25:25.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878dffb700 1 -- 172.21.15.100:0/2054589019 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878800a2e0 con 0x7f8790159f20 2024-08-17T02:25:25.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.848+0000 7f878dffb700 1 -- 172.21.15.100:0/2054589019 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800cd50 con 0x7f8790159f20 2024-08-17T02:25:25.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 -- 172.21.15.100:0/2054589019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 msgr2=0x7f879006e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 --2- 172.21.15.100:0/2054589019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 0x7f879006e350 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8790064cb0 tx=0x7f878800a950 comp rx=0 tx=0).stop 2024-08-17T02:25:25.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 -- 172.21.15.100:0/2054589019 shutdown_connections 2024-08-17T02:25:25.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 --2- 172.21.15.100:0/2054589019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 0x7f879006e350 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 --2- 172.21.15.100:0/2054589019 >> v2:172.21.15.123:3300/0 conn(0x7f8790155520 0x7f87900cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 -- 172.21.15.100:0/2054589019 >> 172.21.15.100:0/2054589019 conn(0x7f8790066e00 msgr2=0x7f8790067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:25.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 -- 172.21.15.100:0/2054589019 shutdown_connections 2024-08-17T02:25:25.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 -- 172.21.15.100:0/2054589019 wait complete. 2024-08-17T02:25:25.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 Processor -- start 2024-08-17T02:25:25.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.852+0000 7f878ffff700 1 -- start start 2024-08-17T02:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8790155520 0x7f879014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87901593c0 0x7f879014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 0x7f87900aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878ffff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879016a0c0 con 0x7f8790159f20 2024-08-17T02:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87901694d0 con 0x7f87901593c0 2024-08-17T02:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8790169790 con 0x7f8790155520 2024-08-17T02:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 0x7f87900aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878effd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8790155520 0x7f879014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 0x7f87900aa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48040/0 (socket says 172.21.15.100:48040) 2024-08-17T02:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 -- 172.21.15.100:0/217749985 learned_addr learned my addr 172.21.15.100:0/217749985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878e7fc700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87901593c0 0x7f879014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 -- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8790155520 msgr2=0x7f879014ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8790155520 0x7f879014ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 -- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87901593c0 msgr2=0x7f879014fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87901593c0 0x7f879014fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 -- 172.21.15.100:0/217749985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878800abf0 con 0x7f8790159f20 2024-08-17T02:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878effd700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8790155520 0x7f879014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878f7fe700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 0x7f87900aa830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f879006f5a0 tx=0x7f8788019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880082a0 con 0x7f8790159f20 2024-08-17T02:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788008e60 con 0x7f8790159f20 2024-08-17T02:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8790169a70 con 0x7f8790159f20 2024-08-17T02:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878801c6a0 con 0x7f8790159f20 2024-08-17T02:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.856+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879014cfb0 con 0x7f8790159f20 2024-08-17T02:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8788003040 con 0x7f8790159f20 2024-08-17T02:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f879014d1d0 con 0x7f8790159f20 2024-08-17T02:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f8776ffd700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f877c060860 0x7f877c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f878807df50 con 0x7f8790159f20 2024-08-17T02:25:25.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f8776ffd700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f877c066750 0x7f877c068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:25.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 --> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f877c069210 con 0x7f877c066750 2024-08-17T02:25:25.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87880804b0 con 0x7f8790159f20 2024-08-17T02:25:25.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f878effd700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f877c060860 0x7f877c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f878e7fc700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f877c066750 0x7f877c068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:25.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f878effd700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f877c060860 0x7f877c062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8780011480 tx=0x7f87800091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.860+0000 7f878e7fc700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f877c066750 0x7f877c068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.864+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 <== osd.1 v2:172.21.15.100:6824/24273 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f877c069210 con 0x7f877c066750 2024-08-17T02:25:25.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 --> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f87900644b0 con 0x7f877c066750 2024-08-17T02:25:25.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f8776ffd700 1 -- 172.21.15.100:0/217749985 <== osd.1 v2:172.21.15.100:6824/24273 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f87900644b0 con 0x7f877c066750 2024-08-17T02:25:25.881 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:25:25.882 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:25:25.882 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:25:25.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f877c066750 msgr2=0x7f877c068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.100:6824/24273,v1:172.21.15.100:6825/24273] conn(0x7f877c066750 0x7f877c068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f877c060860 msgr2=0x7f877c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f877c060860 0x7f877c062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8780011480 tx=0x7f87800091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:25.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 msgr2=0x7f87900aa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 --2- 172.21.15.100:0/217749985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8790159f20 0x7f87900aa830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f879006f5a0 tx=0x7f8788019000 comp rx=0 tx=0).stop 2024-08-17T02:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878f7fe700 1 -- 172.21.15.100:0/217749985 reap_dead start 2024-08-17T02:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 shutdown_connections 2024-08-17T02:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 >> 172.21.15.100:0/217749985 conn(0x7f8790066e00 msgr2=0x7f87900671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 shutdown_connections 2024-08-17T02:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:25.880+0000 7f878ffff700 1 -- 172.21.15.100:0/217749985 wait complete. 2024-08-17T02:25:25.892 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-08-17T02:25:26.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.000+0000 7fd0ddab4700 1 Processor -- start 2024-08-17T02:25:26.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.000+0000 7fd0ddab4700 1 -- start start 2024-08-17T02:25:26.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd0d8159bd0 0x7fd0d80aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d80af360 0x7fd0d8155770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d80ca400 con 0x7fd0d8159fa0 2024-08-17T02:25:26.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d80c97f0 con 0x7fd0d8159bd0 2024-08-17T02:25:26.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d80c9ad0 con 0x7fd0d80af360 2024-08-17T02:25:26.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0dd2b3700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd0d8159fa0 0x7fd0d8155cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56010/0 (socket says 172.21.15.100:56010) 2024-08-17T02:25:26.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0dd2b3700 1 -- 172.21.15.100:0/3859879119 learned_addr learned my addr 172.21.15.100:0/3859879119 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d7fff700 1 --2- 172.21.15.100:0/3859879119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d80af360 0x7fd0d8155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0dcab2700 1 --2- 172.21.15.100:0/3859879119 >> v2:172.21.15.123:3300/0 conn(0x7fd0d8159bd0 0x7fd0d80aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d77fe700 1 -- 172.21.15.100:0/3859879119 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268854110 0 0) 0x7fd0d80ca400 con 0x7fd0d8159fa0 2024-08-17T02:25:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d77fe700 1 -- 172.21.15.100:0/3859879119 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c0003570 con 0x7fd0d8159fa0 2024-08-17T02:25:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d7fff700 1 -- 172.21.15.100:0/3859879119 >> v2:172.21.15.123:3300/0 conn(0x7fd0d8159bd0 msgr2=0x7fd0d80aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d7fff700 1 --2- 172.21.15.100:0/3859879119 >> v2:172.21.15.123:3300/0 conn(0x7fd0d8159bd0 0x7fd0d80aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d7fff700 1 -- 172.21.15.100:0/3859879119 >> v1:172.21.15.100:6789/0 conn(0x7fd0d8159fa0 legacy=0x7fd0d8155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d7fff700 1 -- 172.21.15.100:0/3859879119 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d80c9db0 con 0x7fd0d80af360 2024-08-17T02:25:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d7fff700 1 --2- 172.21.15.100:0/3859879119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d80af360 0x7fd0d8155770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80098e0 tx=0x7fd0c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c8f39334bb3d30 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d77fe700 1 -- 172.21.15.100:0/3859879119 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8011630 con 0x7fd0d80af360 2024-08-17T02:25:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d77fe700 1 -- 172.21.15.100:0/3859879119 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c8011790 con 0x7fd0d80af360 2024-08-17T02:25:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0d77fe700 1 -- 172.21.15.100:0/3859879119 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8011b10 con 0x7fd0d80af360 2024-08-17T02:25:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 -- 172.21.15.100:0/3859879119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d80af360 msgr2=0x7fd0d8155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/3859879119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d80af360 0x7fd0d8155770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80098e0 tx=0x7fd0c80091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 -- 172.21.15.100:0/3859879119 shutdown_connections 2024-08-17T02:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/3859879119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d80af360 0x7fd0d8155770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/3859879119 >> v2:172.21.15.123:3300/0 conn(0x7fd0d8159bd0 0x7fd0d80aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.004+0000 7fd0ddab4700 1 -- 172.21.15.100:0/3859879119 >> 172.21.15.100:0/3859879119 conn(0x7fd0d806e100 msgr2=0x7fd0d80b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.008+0000 7fd0ddab4700 1 -- 172.21.15.100:0/3859879119 shutdown_connections 2024-08-17T02:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.008+0000 7fd0ddab4700 1 -- 172.21.15.100:0/3859879119 wait complete. 2024-08-17T02:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.008+0000 7fd0ddab4700 1 Processor -- start 2024-08-17T02:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.008+0000 7fd0ddab4700 1 -- start start 2024-08-17T02:25:26.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d80af360 0x7fd0d814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d8159bd0 0x7fd0d806a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0d8159fa0 0x7fd0d8154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d814fba0 con 0x7fd0d80af360 2024-08-17T02:25:26.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d8068a70 con 0x7fd0d8159fa0 2024-08-17T02:25:26.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d8068cc0 con 0x7fd0d8159bd0 2024-08-17T02:25:26.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d80af360 0x7fd0d814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d80af360 0x7fd0d814f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48052/0 (socket says 172.21.15.100:48052) 2024-08-17T02:25:26.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 -- 172.21.15.100:0/121630790 learned_addr learned my addr 172.21.15.100:0/121630790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:26.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dd2b3700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0d8159fa0 0x7fd0d8154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0d7fff700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d8159bd0 0x7fd0d806a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 -- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d8159bd0 msgr2=0x7fd0d806a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d8159bd0 0x7fd0d806a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 -- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0d8159fa0 msgr2=0x7fd0d8154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0d8159fa0 0x7fd0d8154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 -- 172.21.15.100:0/121630790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c8009420 con 0x7fd0d80af360 2024-08-17T02:25:26.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0dcab2700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d80af360 0x7fd0d814f230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00ec80 tx=0x7fd0cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0d4ff9700 1 -- 172.21.15.100:0/121630790 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00c850 con 0x7fd0d80af360 2024-08-17T02:25:26.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0d4ff9700 1 -- 172.21.15.100:0/121630790 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc00c9b0 con 0x7fd0d80af360 2024-08-17T02:25:26.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d8068fa0 con 0x7fd0d80af360 2024-08-17T02:25:26.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0d4ff9700 1 -- 172.21.15.100:0/121630790 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00cd50 con 0x7fd0d80af360 2024-08-17T02:25:26.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d8069560 con 0x7fd0d80af360 2024-08-17T02:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.012+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d80638f0 con 0x7fd0d80af360 2024-08-17T02:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.016+0000 7fd0d4ff9700 1 -- 172.21.15.100:0/121630790 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd0cc013680 con 0x7fd0d80af360 2024-08-17T02:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.016+0000 7fd0d4ff9700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd0c4060860 0x7fd0c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.016+0000 7fd0d4ff9700 1 -- 172.21.15.100:0/121630790 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fd0cc07cb30 con 0x7fd0d80af360 2024-08-17T02:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.016+0000 7fd0d7fff700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd0c4060860 0x7fd0c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.016+0000 7fd0d7fff700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd0c4060860 0x7fd0c4062d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80091e0 tx=0x7fd0c8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.020+0000 7fd0d4ff9700 1 -- 172.21.15.100:0/121630790 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd0cc04d220 con 0x7fd0d80af360 2024-08-17T02:25:26.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0d814fd00 con 0x7fd0c4060860 2024-08-17T02:25:26.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0d4ff9700 1 -- 172.21.15.100:0/121630790 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd0d814fd00 con 0x7fd0c4060860 2024-08-17T02:25:26.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd0c4060860 msgr2=0x7fd0c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd0c4060860 0x7fd0c4062d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80091e0 tx=0x7fd0c8017000 comp rx=0 tx=0).stop 2024-08-17T02:25:26.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d80af360 msgr2=0x7fd0d814f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d80af360 0x7fd0d814f230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00ec80 tx=0x7fd0cc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 shutdown_connections 2024-08-17T02:25:26.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd0c4060860 0x7fd0c4062d10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0d8159fa0 0x7fd0d8154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0d8159bd0 0x7fd0d806a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 --2- 172.21.15.100:0/121630790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d80af360 0x7fd0d814f230 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 >> 172.21.15.100:0/121630790 conn(0x7fd0d806e100 msgr2=0x7fd0d80b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 shutdown_connections 2024-08-17T02:25:26.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.188+0000 7fd0ddab4700 1 -- 172.21.15.100:0/121630790 wait complete. 2024-08-17T02:25:26.193 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 1 to deep-scrub 2024-08-17T02:25:26.203 INFO:tasks.ceph:Scrubbing osd.2 2024-08-17T02:25:26.204 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:25:26.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.316+0000 7fbd0bfff700 1 Processor -- start 2024-08-17T02:25:26.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.316+0000 7fbd0bfff700 1 -- start start 2024-08-17T02:25:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbd0c156d60 0x7fbd0c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c0ae9e0 0x7fbd0c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0bfff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbd0c157130 0x7fbd0c0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0bfff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c0afad0 con 0x7fbd0c157130 2024-08-17T02:25:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0bfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c0aeec0 con 0x7fbd0c156d60 2024-08-17T02:25:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0af1a0 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbd0c157130 0x7fbd0c0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbd0c157130 0x7fbd0c0b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48064/0 (socket says 172.21.15.100:48064) 2024-08-17T02:25:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 -- 172.21.15.100:0/768958798 learned_addr learned my addr 172.21.15.100:0/768958798 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0affd700 1 --2- 172.21.15.100:0/768958798 >> v2:172.21.15.123:3300/0 conn(0x7fbd0c156d60 0x7fbd0c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd027fc700 1 --2- 172.21.15.100:0/768958798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c0ae9e0 0x7fbd0c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 -- 172.21.15.100:0/768958798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c0ae9e0 msgr2=0x7fbd0c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 --2- 172.21.15.100:0/768958798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c0ae9e0 0x7fbd0c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 -- 172.21.15.100:0/768958798 >> v2:172.21.15.123:3300/0 conn(0x7fbd0c156d60 msgr2=0x7fbd0c0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 --2- 172.21.15.100:0/768958798 >> v2:172.21.15.123:3300/0 conn(0x7fbd0c156d60 0x7fbd0c0b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 -- 172.21.15.100:0/768958798 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0af480 con 0x7fbd0c157130 2024-08-17T02:25:26.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0b7fe700 1 --2- 172.21.15.100:0/768958798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c157130 0x7fbd0c0b3000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c06ede0 tx=0x7fbd04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b9a35b704360b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0a7fc700 1 -- 172.21.15.100:0/768958798 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400e070 con 0x7fbd0c157130 2024-08-17T02:25:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0a7fc700 1 -- 172.21.15.100:0/768958798 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0400b970 con 0x7fbd0c157130 2024-08-17T02:25:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.320+0000 7fbd0a7fc700 1 -- 172.21.15.100:0/768958798 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd040092e0 con 0x7fbd0c157130 2024-08-17T02:25:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 -- 172.21.15.100:0/768958798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c157130 msgr2=0x7fbd0c0b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 --2- 172.21.15.100:0/768958798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c157130 0x7fbd0c0b3000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c06ede0 tx=0x7fbd04009840 comp rx=0 tx=0).stop 2024-08-17T02:25:26.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 -- 172.21.15.100:0/768958798 shutdown_connections 2024-08-17T02:25:26.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 --2- 172.21.15.100:0/768958798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c157130 0x7fbd0c0b3000 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 --2- 172.21.15.100:0/768958798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c0ae9e0 0x7fbd0c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 --2- 172.21.15.100:0/768958798 >> v2:172.21.15.123:3300/0 conn(0x7fbd0c156d60 0x7fbd0c0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 -- 172.21.15.100:0/768958798 >> 172.21.15.100:0/768958798 conn(0x7fbd0c0cdea0 msgr2=0x7fbd0c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 -- 172.21.15.100:0/768958798 shutdown_connections 2024-08-17T02:25:26.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.324+0000 7fbd0bfff700 1 -- 172.21.15.100:0/768958798 wait complete. 2024-08-17T02:25:26.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 Processor -- start 2024-08-17T02:25:26.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 -- start start 2024-08-17T02:25:26.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c0ae9e0 0x7fbd0c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c156d60 0x7fbd0c0c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd0c157130 0x7fbd0c151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c153010 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1532b0 con 0x7fbd0c157130 2024-08-17T02:25:26.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c153590 con 0x7fbd0c156d60 2024-08-17T02:25:26.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c0ae9e0 0x7fbd0c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c0ae9e0 0x7fbd0c158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48070/0 (socket says 172.21.15.100:48070) 2024-08-17T02:25:26.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 -- 172.21.15.100:0/1110351220 learned_addr learned my addr 172.21.15.100:0/1110351220 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:26.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0b7fe700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd0c157130 0x7fbd0c151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd027fc700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c156d60 0x7fbd0c0c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 -- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c156d60 msgr2=0x7fbd0c0c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd0c156d60 0x7fbd0c0c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 -- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd0c157130 msgr2=0x7fbd0c151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd0c157130 0x7fbd0c151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 -- 172.21.15.100:0/1110351220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04009ae0 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0affd700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c0ae9e0 0x7fbd0c158e90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00ec80 tx=0x7fbcfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00c850 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc00c9b0 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.328+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c153870 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00cd50 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c0c4530 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbcfc013680 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd0c0644b0 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd037fe700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbcf0060860 0x7fbcf0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fbcfc07cc70 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd037fe700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7fbcf0066750 0x7fbcf0068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 --> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbcf0069210 con 0x7fbcf0066750 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcfc081220 con 0x7fbd0c0ae9e0 2024-08-17T02:25:26.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd027fc700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbcf0060860 0x7fbcf0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd0b7fe700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7fbcf0066750 0x7fbcf0068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd0b7fe700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7fbcf0066750 0x7fbcf0068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.332+0000 7fbd027fc700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbcf0060860 0x7fbcf0062d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0ca860 tx=0x7fbcf40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.336+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 <== osd.2 v2:172.21.15.100:6809/24272 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbcf0069210 con 0x7fbcf0066750 2024-08-17T02:25:26.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 --> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbd0c065610 con 0x7fbcf0066750 2024-08-17T02:25:26.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd037fe700 1 -- 172.21.15.100:0/1110351220 <== osd.2 v2:172.21.15.100:6809/24272 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbd0c065610 con 0x7fbcf0066750 2024-08-17T02:25:26.355 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:25:26.355 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:25:26.355 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:25:26.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7fbcf0066750 msgr2=0x7fbcf0068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.100:6809/24272,v1:172.21.15.100:6810/24272] conn(0x7fbcf0066750 0x7fbcf0068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbcf0060860 msgr2=0x7fbcf0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fbcf0060860 0x7fbcf0062d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0ca860 tx=0x7fbcf40091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c0ae9e0 msgr2=0x7fbd0c158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 --2- 172.21.15.100:0/1110351220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd0c0ae9e0 0x7fbd0c158e90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00ec80 tx=0x7fbcfc0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0b7fe700 1 -- 172.21.15.100:0/1110351220 reap_dead start 2024-08-17T02:25:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 shutdown_connections 2024-08-17T02:25:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 >> 172.21.15.100:0/1110351220 conn(0x7fbd0c0cdea0 msgr2=0x7fbd0c0cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 shutdown_connections 2024-08-17T02:25:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.352+0000 7fbd0bfff700 1 -- 172.21.15.100:0/1110351220 wait complete. 2024-08-17T02:25:26.367 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-08-17T02:25:26.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.480+0000 7f37bab6c700 1 Processor -- start 2024-08-17T02:25:26.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.480+0000 7f37bab6c700 1 -- start start 2024-08-17T02:25:26.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f37b4158240 0x7f37b40c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b40ca930 con 0x7f37b40c4270 2024-08-17T02:25:26.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b40cc500 con 0x7f37b4157760 2024-08-17T02:25:26.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b40c8cd0 con 0x7f37b4158240 2024-08-17T02:25:26.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b3fff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f37b40c4270 0x7f37b415ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56022/0 (socket says 172.21.15.100:56022) 2024-08-17T02:25:26.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b3fff700 1 -- 172.21.15.100:0/1159742880 learned_addr learned my addr 172.21.15.100:0/1159742880 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:26.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b9109700 1 --2- 172.21.15.100:0/1159742880 >> v2:172.21.15.123:3300/0 conn(0x7f37b4158240 0x7f37b40c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b37fe700 1 -- 172.21.15.100:0/1159742880 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590316903 0 0) 0x7f37b40ca930 con 0x7f37b40c4270 2024-08-17T02:25:26.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b37fe700 1 -- 172.21.15.100:0/1159742880 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f379c003570 con 0x7f37b40c4270 2024-08-17T02:25:26.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b9109700 1 -- 172.21.15.100:0/1159742880 >> v1:172.21.15.123:6789/0 conn(0x7f37b4157760 legacy=0x7f37b4157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b9109700 1 -- 172.21.15.100:0/1159742880 >> v1:172.21.15.100:6789/0 conn(0x7f37b40c4270 legacy=0x7f37b415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b9109700 1 -- 172.21.15.100:0/1159742880 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40cc780 con 0x7f37b4158240 2024-08-17T02:25:26.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b9109700 1 --2- 172.21.15.100:0/1159742880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 0x7f37b40c8790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009e60 tx=0x7f37ac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d5d1eb883bec8f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b37fe700 1 -- 172.21.15.100:0/1159742880 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00e070 con 0x7f37b4158240 2024-08-17T02:25:26.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b37fe700 1 -- 172.21.15.100:0/1159742880 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00bb60 con 0x7f37b4158240 2024-08-17T02:25:26.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37b37fe700 1 -- 172.21.15.100:0/1159742880 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac012430 con 0x7f37b4158240 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 -- 172.21.15.100:0/1159742880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 msgr2=0x7f37b40c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 --2- 172.21.15.100:0/1159742880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 0x7f37b40c8790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009e60 tx=0x7f37ac009780 comp rx=0 tx=0).stop 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 -- 172.21.15.100:0/1159742880 shutdown_connections 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 --2- 172.21.15.100:0/1159742880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 0x7f37b40c8790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 -- 172.21.15.100:0/1159742880 >> 172.21.15.100:0/1159742880 conn(0x7f37b40caae0 msgr2=0x7f37b40caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 -- 172.21.15.100:0/1159742880 shutdown_connections 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.484+0000 7f37bab6c700 1 -- 172.21.15.100:0/1159742880 wait complete. 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 Processor -- start 2024-08-17T02:25:26.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 -- start start 2024-08-17T02:25:26.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37b40c4270 0x7f37b4151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4157760 0x7f37b414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 0x7f37b414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4154bc0 con 0x7f37b4157760 2024-08-17T02:25:26.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b0610 con 0x7f37b4158240 2024-08-17T02:25:26.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b0050 con 0x7f37b40c4270 2024-08-17T02:25:26.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4157760 0x7f37b414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4157760 0x7f37b414f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48076/0 (socket says 172.21.15.100:48076) 2024-08-17T02:25:26.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b9109700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 0x7f37b414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 -- 172.21.15.100:0/2135855271 learned_addr learned my addr 172.21.15.100:0/2135855271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:26.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b8908700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37b40c4270 0x7f37b4151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 -- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37b40c4270 msgr2=0x7f37b4151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37b40c4270 0x7f37b4151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 -- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 msgr2=0x7f37b414fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 0x7f37b414fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 -- 172.21.15.100:0/2135855271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac009a20 con 0x7f37b4157760 2024-08-17T02:25:26.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b3fff700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4157760 0x7f37b414f830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c5c0 tx=0x7f37a8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b0ff9700 1 -- 172.21.15.100:0/2135855271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8005b30 con 0x7f37b4157760 2024-08-17T02:25:26.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b0ff9700 1 -- 172.21.15.100:0/2135855271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a8005c90 con 0x7f37b4157760 2024-08-17T02:25:26.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b40b0330 con 0x7f37b4157760 2024-08-17T02:25:26.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37b0ff9700 1 -- 172.21.15.100:0/2135855271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8007320 con 0x7f37b4157760 2024-08-17T02:25:26.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.488+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b41522e0 con 0x7f37b4157760 2024-08-17T02:25:26.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.492+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b41525c0 con 0x7f37b4157760 2024-08-17T02:25:26.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.492+0000 7f37b0ff9700 1 -- 172.21.15.100:0/2135855271 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f37a801b3b0 con 0x7f37b4157760 2024-08-17T02:25:26.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.492+0000 7f37b0ff9700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f37a0060860 0x7f37a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.492+0000 7f37b0ff9700 1 -- 172.21.15.100:0/2135855271 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f37a807bd60 con 0x7f37b4157760 2024-08-17T02:25:26.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.492+0000 7f37b8908700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f37a0060860 0x7f37a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.492+0000 7f37b8908700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f37a0060860 0x7f37a0062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000c00 tx=0x7f37a4007ea0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.500+0000 7f37b0ff9700 1 -- 172.21.15.100:0/2135855271 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f37a804c450 con 0x7f37b4157760 2024-08-17T02:25:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.660+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f37b414f530 con 0x7f37a0060860 2024-08-17T02:25:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.660+0000 7f37b0ff9700 1 -- 172.21.15.100:0/2135855271 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f37b414f530 con 0x7f37a0060860 2024-08-17T02:25:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f37a0060860 msgr2=0x7f37a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f37a0060860 0x7f37a0062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000c00 tx=0x7f37a4007ea0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4157760 msgr2=0x7f37b414f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4157760 0x7f37b414f830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c5c0 tx=0x7f37a8007e00 comp rx=0 tx=0).stop 2024-08-17T02:25:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 shutdown_connections 2024-08-17T02:25:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f37a0060860 0x7f37a0062d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b4158240 0x7f37b414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4157760 0x7f37b414f830 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 --2- 172.21.15.100:0/2135855271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37b40c4270 0x7f37b4151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 >> 172.21.15.100:0/2135855271 conn(0x7f37b40caae0 msgr2=0x7f37b406d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 shutdown_connections 2024-08-17T02:25:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.664+0000 7f37bab6c700 1 -- 172.21.15.100:0/2135855271 wait complete. 2024-08-17T02:25:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 2 to deep-scrub 2024-08-17T02:25:26.677 INFO:tasks.ceph:Scrubbing osd.3 2024-08-17T02:25:26.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:25:26.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.788+0000 7f805618e700 1 Processor -- start 2024-08-17T02:25:26.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.788+0000 7f805618e700 1 -- start start 2024-08-17T02:25:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805618e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 0x7f80500cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805618e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80500cbf40 0x7f80500b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805618e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8050067700 con 0x7f80500b2700 2024-08-17T02:25:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805618e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80500cc390 con 0x7f80500cbf40 2024-08-17T02:25:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805618e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80500b2b80 con 0x7f8050140790 2024-08-17T02:25:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805498b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80500cbf40 0x7f80500b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805598d700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f80500b2700 0x7f805015aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56038/0 (socket says 172.21.15.100:56038) 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805598d700 1 -- 172.21.15.100:0/281811532 learned_addr learned my addr 172.21.15.100:0/281811532 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805518c700 1 --2- 172.21.15.100:0/281811532 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 0x7f80500cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805498b700 1 -- 172.21.15.100:0/281811532 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 msgr2=0x7f80500cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805498b700 1 --2- 172.21.15.100:0/281811532 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 0x7f80500cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805498b700 1 -- 172.21.15.100:0/281811532 >> v1:172.21.15.100:6789/0 conn(0x7f80500b2700 legacy=0x7f805015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f8047fff700 1 -- 172.21.15.100:0/281811532 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823393540 0 0) 0x7f8050067700 con 0x7f80500b2700 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805498b700 1 -- 172.21.15.100:0/281811532 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80500af900 con 0x7f80500cbf40 2024-08-17T02:25:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f805498b700 1 --2- 172.21.15.100:0/281811532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500cbf40 0x7f80500b21c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f80400098e0 tx=0x7f80400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe7e832a4de09c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f8047fff700 1 -- 172.21.15.100:0/281811532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804000b640 con 0x7f80500cbf40 2024-08-17T02:25:26.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f8047fff700 1 -- 172.21.15.100:0/281811532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804000b7a0 con 0x7f80500cbf40 2024-08-17T02:25:26.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.792+0000 7f8047fff700 1 -- 172.21.15.100:0/281811532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804000bb40 con 0x7f80500cbf40 2024-08-17T02:25:26.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- 172.21.15.100:0/281811532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500cbf40 msgr2=0x7f80500b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 --2- 172.21.15.100:0/281811532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500cbf40 0x7f80500b21c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f80400098e0 tx=0x7f80400091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- 172.21.15.100:0/281811532 shutdown_connections 2024-08-17T02:25:26.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 --2- 172.21.15.100:0/281811532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500cbf40 0x7f80500b21c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 --2- 172.21.15.100:0/281811532 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 0x7f80500cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- 172.21.15.100:0/281811532 >> 172.21.15.100:0/281811532 conn(0x7f805006a700 msgr2=0x7f80500aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- 172.21.15.100:0/281811532 shutdown_connections 2024-08-17T02:25:26.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- 172.21.15.100:0/281811532 wait complete. 2024-08-17T02:25:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 Processor -- start 2024-08-17T02:25:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- start start 2024-08-17T02:25:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500b2700 0x7f805013e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80500cbf40 0x7f80500c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 0x7f80500c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80500c3d40 con 0x7f80500b2700 2024-08-17T02:25:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80500c4020 con 0x7f80500cbf40 2024-08-17T02:25:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805618e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80500c4300 con 0x7f8050140790 2024-08-17T02:25:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805518c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500b2700 0x7f805013e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805518c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500b2700 0x7f805013e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48084/0 (socket says 172.21.15.100:48084) 2024-08-17T02:25:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805518c700 1 -- 172.21.15.100:0/313315002 learned_addr learned my addr 172.21.15.100:0/313315002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805598d700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 0x7f80500c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.796+0000 7f805498b700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80500cbf40 0x7f80500c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805518c700 1 -- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 msgr2=0x7f80500c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805518c700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8050140790 0x7f80500c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805518c700 1 -- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80500cbf40 msgr2=0x7f80500c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805518c700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80500cbf40 0x7f80500c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805518c700 1 -- 172.21.15.100:0/313315002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040009420 con 0x7f80500b2700 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805518c700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500b2700 0x7f805013e8c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f804c00e930 tx=0x7f804c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c00eed0 con 0x7f80500b2700 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c00c850 con 0x7f80500b2700 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80500c45e0 con 0x7f80500b2700 2024-08-17T02:25:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c00cbf0 con 0x7f80500b2700 2024-08-17T02:25:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80500c8180 con 0x7f80500b2700 2024-08-17T02:25:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80500644b0 con 0x7f80500b2700 2024-08-17T02:25:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f804c013470 con 0x7f80500b2700 2024-08-17T02:25:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8034060860 0x7f8034062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f804c07cc00 con 0x7f80500b2700 2024-08-17T02:25:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805498b700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8034060860 0x7f8034062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f80340665e0 0x7f80340689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 --> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80340690a0 con 0x7f80340665e0 2024-08-17T02:25:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f804c081220 con 0x7f80500b2700 2024-08-17T02:25:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805598d700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f80340665e0 0x7f80340689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.800+0000 7f805598d700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f80340665e0 0x7f80340689f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.804+0000 7f805498b700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8034060860 0x7f8034062d10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80400093f0 tx=0x7f8040017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.804+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 <== osd.3 v2:172.21.15.100:6808/24274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f80340690a0 con 0x7f80340665e0 2024-08-17T02:25:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 --> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f805013ff80 con 0x7f80340665e0 2024-08-17T02:25:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f80457fa700 1 -- 172.21.15.100:0/313315002 <== osd.3 v2:172.21.15.100:6808/24274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f805013ff80 con 0x7f80340665e0 2024-08-17T02:25:26.819 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:25:26.819 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:25:26.819 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:25:26.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f80340665e0 msgr2=0x7f80340689f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.100:6808/24274,v1:172.21.15.100:6811/24274] conn(0x7f80340665e0 0x7f80340689f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8034060860 msgr2=0x7f8034062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8034060860 0x7f8034062d10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80400093f0 tx=0x7f8040017000 comp rx=0 tx=0).stop 2024-08-17T02:25:26.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500b2700 msgr2=0x7f805013e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 --2- 172.21.15.100:0/313315002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80500b2700 0x7f805013e8c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f804c00e930 tx=0x7f804c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805598d700 1 -- 172.21.15.100:0/313315002 reap_dead start 2024-08-17T02:25:26.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 shutdown_connections 2024-08-17T02:25:26.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 >> 172.21.15.100:0/313315002 conn(0x7f805006a700 msgr2=0x7f80500acbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 shutdown_connections 2024-08-17T02:25:26.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.816+0000 7f805618e700 1 -- 172.21.15.100:0/313315002 wait complete. 2024-08-17T02:25:26.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-08-17T02:25:26.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30aad89700 1 Processor -- start 2024-08-17T02:25:26.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30aad89700 1 -- start start 2024-08-17T02:25:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30aad89700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40ae2a0 0x7f30a40ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30aad89700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30a40b0f10 0x7f30a40a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30aad89700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a40a9160 con 0x7f30a415a620 2024-08-17T02:25:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30aad89700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a40aff10 con 0x7f30a40b0f10 2024-08-17T02:25:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30aad89700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40b1680 con 0x7f30a40ae2a0 2024-08-17T02:25:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30a9326700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30a40b0f10 0x7f30a40a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30a9326700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30a40b0f10 0x7f30a40a95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48086/0 (socket says 172.21.15.100:48086) 2024-08-17T02:25:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30a9326700 1 -- 172.21.15.100:0/1281853846 learned_addr learned my addr 172.21.15.100:0/1281853846 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30a3fff700 1 --2- 172.21.15.100:0/1281853846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40ae2a0 0x7f30a40ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30a9326700 1 -- 172.21.15.100:0/1281853846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40ae2a0 msgr2=0x7f30a40ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30a9326700 1 --2- 172.21.15.100:0/1281853846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40ae2a0 0x7f30a40ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30a9326700 1 -- 172.21.15.100:0/1281853846 >> v1:172.21.15.123:6789/0 conn(0x7f30a415a620 legacy=0x7f30a40adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.940+0000 7f30a9326700 1 -- 172.21.15.100:0/1281853846 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40b1960 con 0x7f30a40b0f10 2024-08-17T02:25:26.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30a9326700 1 --2- 172.21.15.100:0/1281853846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40b0f10 0x7f30a40a95f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f309c0099b0 tx=0x7f309c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c22652efbb90da server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30a37fe700 1 -- 172.21.15.100:0/1281853846 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c009ea0 con 0x7f30a40b0f10 2024-08-17T02:25:26.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30a37fe700 1 -- 172.21.15.100:0/1281853846 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c00b860 con 0x7f30a40b0f10 2024-08-17T02:25:26.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30a37fe700 1 -- 172.21.15.100:0/1281853846 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00bc00 con 0x7f30a40b0f10 2024-08-17T02:25:26.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 -- 172.21.15.100:0/1281853846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40b0f10 msgr2=0x7f30a40a95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 --2- 172.21.15.100:0/1281853846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40b0f10 0x7f30a40a95f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f309c0099b0 tx=0x7f309c009250 comp rx=0 tx=0).stop 2024-08-17T02:25:26.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 -- 172.21.15.100:0/1281853846 shutdown_connections 2024-08-17T02:25:26.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 --2- 172.21.15.100:0/1281853846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40b0f10 0x7f30a40a95f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 --2- 172.21.15.100:0/1281853846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40ae2a0 0x7f30a40ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 -- 172.21.15.100:0/1281853846 >> 172.21.15.100:0/1281853846 conn(0x7f30a4067810 msgr2=0x7f30a4067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:26.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 -- 172.21.15.100:0/1281853846 shutdown_connections 2024-08-17T02:25:26.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 -- 172.21.15.100:0/1281853846 wait complete. 2024-08-17T02:25:26.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 Processor -- start 2024-08-17T02:25:26.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.944+0000 7f30aad89700 1 -- start start 2024-08-17T02:25:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30aad89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a40ae2a0 0x7f30a41673d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30aad89700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40b0f10 0x7f30a40ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30aad89700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a415a620 0x7f30a40c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30aad89700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40c9530 con 0x7f30a415a620 2024-08-17T02:25:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30aad89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a4096b00 con 0x7f30a40ae2a0 2024-08-17T02:25:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30aad89700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a4096de0 con 0x7f30a40b0f10 2024-08-17T02:25:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a415a620 0x7f30a40c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a415a620 0x7f30a40c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48096/0 (socket says 172.21.15.100:48096) 2024-08-17T02:25:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 -- 172.21.15.100:0/3870972123 learned_addr learned my addr 172.21.15.100:0/3870972123 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a8b25700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a40ae2a0 0x7f30a41673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a3fff700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40b0f10 0x7f30a40ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 -- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40b0f10 msgr2=0x7f30a40ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40b0f10 0x7f30a40ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 -- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a40ae2a0 msgr2=0x7f30a41673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:26.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a40ae2a0 0x7f30a41673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:26.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 -- 172.21.15.100:0/3870972123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0094f0 con 0x7f30a415a620 2024-08-17T02:25:26.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a9326700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a415a620 0x7f30a40c8a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f30a406ccb0 tx=0x7f309c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a0ff9700 1 -- 172.21.15.100:0/3870972123 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00baa0 con 0x7f30a415a620 2024-08-17T02:25:26.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a0ff9700 1 -- 172.21.15.100:0/3870972123 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c025420 con 0x7f30a415a620 2024-08-17T02:25:26.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a40970c0 con 0x7f30a415a620 2024-08-17T02:25:26.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30a0ff9700 1 -- 172.21.15.100:0/3870972123 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00bc00 con 0x7f30a415a620 2024-08-17T02:25:26.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.948+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a4099520 con 0x7f30a415a620 2024-08-17T02:25:26.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.952+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a40645c0 con 0x7f30a415a620 2024-08-17T02:25:26.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.952+0000 7f30a0ff9700 1 -- 172.21.15.100:0/3870972123 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f309c023070 con 0x7f30a415a620 2024-08-17T02:25:26.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.952+0000 7f30a0ff9700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3090060860 0x7f3090062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:26.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.952+0000 7f30a0ff9700 1 -- 172.21.15.100:0/3870972123 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f309c07dfa0 con 0x7f30a415a620 2024-08-17T02:25:26.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.952+0000 7f30a8b25700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3090060860 0x7f3090062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:26.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.952+0000 7f30a8b25700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3090060860 0x7f3090062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30980039e0 tx=0x7f3098007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:26.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:26.960+0000 7f30a0ff9700 1 -- 172.21.15.100:0/3870972123 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f309c0ae480 con 0x7f30a415a620 2024-08-17T02:25:27.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f30a406b540 con 0x7f3090060860 2024-08-17T02:25:27.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30a0ff9700 1 -- 172.21.15.100:0/3870972123 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f30a406b540 con 0x7f3090060860 2024-08-17T02:25:27.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3090060860 msgr2=0x7f3090062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3090060860 0x7f3090062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30980039e0 tx=0x7f3098007ec0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a415a620 msgr2=0x7f30a40c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a415a620 0x7f30a40c8a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f30a406ccb0 tx=0x7f309c017000 comp rx=0 tx=0).stop 2024-08-17T02:25:27.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 shutdown_connections 2024-08-17T02:25:27.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3090060860 0x7f3090062d10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a415a620 0x7f30a40c8a30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30a40b0f10 0x7f30a40ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 --2- 172.21.15.100:0/3870972123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a40ae2a0 0x7f30a41673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 >> 172.21.15.100:0/3870972123 conn(0x7f30a4067810 msgr2=0x7f30a4157860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:27.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 shutdown_connections 2024-08-17T02:25:27.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.124+0000 7f30aad89700 1 -- 172.21.15.100:0/3870972123 wait complete. 2024-08-17T02:25:27.130 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 3 to deep-scrub 2024-08-17T02:25:27.140 INFO:tasks.ceph:Scrubbing osd.4 2024-08-17T02:25:27.140 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:25:27.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d81c700 1 Processor -- start 2024-08-17T02:25:27.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d81c700 1 -- start start 2024-08-17T02:25:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d81c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e8815a3e0 0x7f8e880ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d81c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e8815a7b0 0x7f8e880ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d81c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8e880686a0 0x7f8e88068b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d81c700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e88069060 con 0x7f8e8815a7b0 2024-08-17T02:25:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d81c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e880b0dc0 con 0x7f8e880686a0 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d81c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880b04d0 con 0x7f8e8815a3e0 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e87fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e8815a7b0 0x7f8e880ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e87fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e8815a7b0 0x7f8e880ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48098/0 (socket says 172.21.15.100:48098) 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e87fff700 1 -- 172.21.15.100:0/187017174 learned_addr learned my addr 172.21.15.100:0/187017174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d01b700 1 --2- 172.21.15.100:0/187017174 >> v2:172.21.15.123:3300/0 conn(0x7f8e880686a0 0x7f8e88068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8c81a700 1 --2- 172.21.15.100:0/187017174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e8815a3e0 0x7f8e880ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e87fff700 1 -- 172.21.15.100:0/187017174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e8815a3e0 msgr2=0x7f8e880ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e87fff700 1 --2- 172.21.15.100:0/187017174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e8815a3e0 0x7f8e880ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e87fff700 1 -- 172.21.15.100:0/187017174 >> v2:172.21.15.123:3300/0 conn(0x7f8e880686a0 msgr2=0x7f8e88068b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e87fff700 1 --2- 172.21.15.100:0/187017174 >> v2:172.21.15.123:3300/0 conn(0x7f8e880686a0 0x7f8e88068b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e87fff700 1 -- 172.21.15.100:0/187017174 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8806e990 con 0x7f8e8815a7b0 2024-08-17T02:25:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.248+0000 7f8e8d01b700 1 --2- 172.21.15.100:0/187017174 >> v2:172.21.15.123:3300/0 conn(0x7f8e880686a0 0x7f8e88068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e87fff700 1 --2- 172.21.15.100:0/187017174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 0x7f8e880ae880 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e8806d940 tx=0x7f8e74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d392057e6108bb93 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e877fe700 1 -- 172.21.15.100:0/187017174 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400b640 con 0x7f8e8815a7b0 2024-08-17T02:25:27.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e877fe700 1 -- 172.21.15.100:0/187017174 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7400b7a0 con 0x7f8e8815a7b0 2024-08-17T02:25:27.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e877fe700 1 -- 172.21.15.100:0/187017174 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400bb60 con 0x7f8e8815a7b0 2024-08-17T02:25:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 -- 172.21.15.100:0/187017174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 msgr2=0x7f8e880ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 --2- 172.21.15.100:0/187017174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 0x7f8e880ae880 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e8806d940 tx=0x7f8e74009250 comp rx=0 tx=0).stop 2024-08-17T02:25:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 -- 172.21.15.100:0/187017174 shutdown_connections 2024-08-17T02:25:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 --2- 172.21.15.100:0/187017174 >> v2:172.21.15.123:3300/0 conn(0x7f8e880686a0 0x7f8e88068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 --2- 172.21.15.100:0/187017174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 0x7f8e880ae880 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 --2- 172.21.15.100:0/187017174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e8815a3e0 0x7f8e880ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 -- 172.21.15.100:0/187017174 >> 172.21.15.100:0/187017174 conn(0x7f8e880ccfa0 msgr2=0x7f8e8806d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 -- 172.21.15.100:0/187017174 shutdown_connections 2024-08-17T02:25:27.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.252+0000 7f8e8d81c700 1 -- 172.21.15.100:0/187017174 wait complete. 2024-08-17T02:25:27.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d81c700 1 Processor -- start 2024-08-17T02:25:27.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d81c700 1 -- start start 2024-08-17T02:25:27.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d81c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e880686a0 0x7f8e88143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d81c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e8815a3e0 0x7f8e880a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d81c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 0x7f8e880a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d81c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880a7570 con 0x7f8e8815a7b0 2024-08-17T02:25:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d81c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880a77c0 con 0x7f8e8815a3e0 2024-08-17T02:25:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d81c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880a7aa0 con 0x7f8e880686a0 2024-08-17T02:25:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d01b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 0x7f8e880a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e87fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e8815a3e0 0x7f8e880a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d01b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 0x7f8e880a6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48112/0 (socket says 172.21.15.100:48112) 2024-08-17T02:25:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d01b700 1 -- 172.21.15.100:0/2707266654 learned_addr learned my addr 172.21.15.100:0/2707266654 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8c81a700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e880686a0 0x7f8e88143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d01b700 1 -- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e880686a0 msgr2=0x7f8e88143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d01b700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e880686a0 0x7f8e88143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d01b700 1 -- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e8815a3e0 msgr2=0x7f8e880a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d01b700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e8815a3e0 0x7f8e880a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.256+0000 7f8e8d01b700 1 -- 172.21.15.100:0/2707266654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80009a20 con 0x7f8e8815a7b0 2024-08-17T02:25:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e8d01b700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 0x7f8e880a6ae0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009f20 tx=0x7f8e8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000cb70 con 0x7f8e8815a7b0 2024-08-17T02:25:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8000f040 con 0x7f8e8815a7b0 2024-08-17T02:25:27.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e740094f0 con 0x7f8e8815a7b0 2024-08-17T02:25:27.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80014480 con 0x7f8e8815a7b0 2024-08-17T02:25:27.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e880a8060 con 0x7f8e8815a7b0 2024-08-17T02:25:27.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e880644b0 con 0x7f8e8815a7b0 2024-08-17T02:25:27.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8e80003020 con 0x7f8e8815a7b0 2024-08-17T02:25:27.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.260+0000 7f8e84ff9700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8e6c060860 0x7f8e6c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f8e8007dbd0 con 0x7f8e8815a7b0 2024-08-17T02:25:27.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e8c81a700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8e6c060860 0x7f8e6c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e84ff9700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f8e6c066750 0x7f8e6c068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 --> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e6c069210 con 0x7f8e6c066750 2024-08-17T02:25:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e8007dfa0 con 0x7f8e8815a7b0 2024-08-17T02:25:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e87fff700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f8e6c066750 0x7f8e6c068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e8c81a700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8e6c060860 0x7f8e6c062d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009c80 tx=0x7f8e7c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e87fff700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f8e6c066750 0x7f8e6c068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.264+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 <== osd.4 v2:172.21.15.123:6803/24439 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8e6c069210 con 0x7f8e6c066750 2024-08-17T02:25:27.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.284+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 --> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8e88069cd0 con 0x7f8e6c066750 2024-08-17T02:25:27.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.284+0000 7f8e84ff9700 1 -- 172.21.15.100:0/2707266654 <== osd.4 v2:172.21.15.123:6803/24439 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8e88069cd0 con 0x7f8e6c066750 2024-08-17T02:25:27.288 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:25:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:25:27.288 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:25:27.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.284+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f8e6c066750 msgr2=0x7f8e6c068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.284+0000 7f8e8d81c700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6803/24439,v1:172.21.15.123:6805/24439] conn(0x7f8e6c066750 0x7f8e6c068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8e6c060860 msgr2=0x7f8e6c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d81c700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8e6c060860 0x7f8e6c062d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009c80 tx=0x7f8e7c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 msgr2=0x7f8e880a6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d81c700 1 --2- 172.21.15.100:0/2707266654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a7b0 0x7f8e880a6ae0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009f20 tx=0x7f8e8000b040 comp rx=0 tx=0).stop 2024-08-17T02:25:27.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d01b700 1 -- 172.21.15.100:0/2707266654 reap_dead start 2024-08-17T02:25:27.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 shutdown_connections 2024-08-17T02:25:27.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 >> 172.21.15.100:0/2707266654 conn(0x7f8e880ccfa0 msgr2=0x7f8e880b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:27.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 shutdown_connections 2024-08-17T02:25:27.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.288+0000 7f8e8d81c700 1 -- 172.21.15.100:0/2707266654 wait complete. 2024-08-17T02:25:27.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-08-17T02:25:27.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f329a1af700 1 Processor -- start 2024-08-17T02:25:27.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f329a1af700 1 -- start start 2024-08-17T02:25:27.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f329a1af700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32940b2390 0x7f32940686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f329a1af700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3294068be0 0x7f3294155800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f329a1af700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32940ae790 con 0x7f3294155430 2024-08-17T02:25:27.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f329a1af700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3294069020 con 0x7f32940b2390 2024-08-17T02:25:27.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f329a1af700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32940adb80 con 0x7f3294068be0 2024-08-17T02:25:27.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3292ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32940b2390 0x7f32940686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3292ffd700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32940b2390 0x7f32940686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48126/0 (socket says 172.21.15.100:48126) 2024-08-17T02:25:27.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3292ffd700 1 -- 172.21.15.100:0/1046639698 learned_addr learned my addr 172.21.15.100:0/1046639698 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:27.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3293fff700 1 --2- 172.21.15.100:0/1046639698 >> v2:172.21.15.123:3300/0 conn(0x7f3294068be0 0x7f3294155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3292ffd700 1 -- 172.21.15.100:0/1046639698 >> v2:172.21.15.123:3300/0 conn(0x7f3294068be0 msgr2=0x7f3294155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:27.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3292ffd700 1 --2- 172.21.15.100:0/1046639698 >> v2:172.21.15.123:3300/0 conn(0x7f3294068be0 0x7f3294155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3292ffd700 1 -- 172.21.15.100:0/1046639698 >> v1:172.21.15.123:6789/0 conn(0x7f3294155430 legacy=0x7f32940af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3292ffd700 1 -- 172.21.15.100:0/1046639698 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940ade60 con 0x7f32940b2390 2024-08-17T02:25:27.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f3292ffd700 1 --2- 172.21.15.100:0/1046639698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32940b2390 0x7f32940686a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f328400bc50 tx=0x7f328400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c268a303c0c4f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f32927fc700 1 -- 172.21.15.100:0/1046639698 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400dbb0 con 0x7f32940b2390 2024-08-17T02:25:27.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f32927fc700 1 -- 172.21.15.100:0/1046639698 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328400dd10 con 0x7f32940b2390 2024-08-17T02:25:27.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.408+0000 7f32927fc700 1 -- 172.21.15.100:0/1046639698 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32840196b0 con 0x7f32940b2390 2024-08-17T02:25:27.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- 172.21.15.100:0/1046639698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32940b2390 msgr2=0x7f32940686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 --2- 172.21.15.100:0/1046639698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32940b2390 0x7f32940686a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f328400bc50 tx=0x7f328400b4f0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- 172.21.15.100:0/1046639698 shutdown_connections 2024-08-17T02:25:27.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 --2- 172.21.15.100:0/1046639698 >> v2:172.21.15.123:3300/0 conn(0x7f3294068be0 0x7f3294155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 --2- 172.21.15.100:0/1046639698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32940b2390 0x7f32940686a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- 172.21.15.100:0/1046639698 >> 172.21.15.100:0/1046639698 conn(0x7f329406e4e0 msgr2=0x7f329406bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:27.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- 172.21.15.100:0/1046639698 shutdown_connections 2024-08-17T02:25:27.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- 172.21.15.100:0/1046639698 wait complete. 2024-08-17T02:25:27.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 Processor -- start 2024-08-17T02:25:27.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- start start 2024-08-17T02:25:27.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3294068be0 0x7f32941401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32940b2390 0x7f329414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3294155430 0x7f329414ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329415a9a0 con 0x7f3294068be0 2024-08-17T02:25:27.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32941406f0 con 0x7f32940b2390 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329414fe60 con 0x7f3294155430 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3294068be0 0x7f32941401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3294068be0 0x7f32941401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48140/0 (socket says 172.21.15.100:48140) 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 -- 172.21.15.100:0/3173677040 learned_addr learned my addr 172.21.15.100:0/3173677040 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f3292ffd700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32940b2390 0x7f329414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f3293fff700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3294155430 0x7f329414ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 -- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3294155430 msgr2=0x7f329414ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3294155430 0x7f329414ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 -- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32940b2390 msgr2=0x7f329414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32940b2390 0x7f329414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 -- 172.21.15.100:0/3173677040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328400b790 con 0x7f3294068be0 2024-08-17T02:25:27.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f3293fff700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3294155430 0x7f329414ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:25:27.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f32937fe700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3294068be0 0x7f32941401b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c5c0 tx=0x7f327c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f328b7fe700 1 -- 172.21.15.100:0/3173677040 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c011590 con 0x7f3294068be0 2024-08-17T02:25:27.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f328b7fe700 1 -- 172.21.15.100:0/3173677040 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c011710 con 0x7f3294068be0 2024-08-17T02:25:27.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329414f250 con 0x7f3294068be0 2024-08-17T02:25:27.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f328b7fe700 1 -- 172.21.15.100:0/3173677040 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c011ad0 con 0x7f3294068be0 2024-08-17T02:25:27.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.412+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329414f810 con 0x7f3294068be0 2024-08-17T02:25:27.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.416+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32940645c0 con 0x7f3294068be0 2024-08-17T02:25:27.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.416+0000 7f328b7fe700 1 -- 172.21.15.100:0/3173677040 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f327c01b3b0 con 0x7f3294068be0 2024-08-17T02:25:27.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.416+0000 7f328b7fe700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3274060860 0x7f3274062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.416+0000 7f328b7fe700 1 -- 172.21.15.100:0/3173677040 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f327c07be70 con 0x7f3294068be0 2024-08-17T02:25:27.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.416+0000 7f3292ffd700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3274060860 0x7f3274062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.416+0000 7f3292ffd700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3274060860 0x7f3274062d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f328400bc50 tx=0x7f328401b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.424+0000 7f328b7fe700 1 -- 172.21.15.100:0/3173677040 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f327c04c560 con 0x7f3294068be0 2024-08-17T02:25:27.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f329406f0a0 con 0x7f3274060860 2024-08-17T02:25:27.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f328b7fe700 1 -- 172.21.15.100:0/3173677040 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f329406f0a0 con 0x7f3274060860 2024-08-17T02:25:27.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3274060860 msgr2=0x7f3274062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3274060860 0x7f3274062d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f328400bc50 tx=0x7f328401b000 comp rx=0 tx=0).stop 2024-08-17T02:25:27.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3294068be0 msgr2=0x7f32941401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3294068be0 0x7f32941401b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c5c0 tx=0x7f327c00c980 comp rx=0 tx=0).stop 2024-08-17T02:25:27.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 shutdown_connections 2024-08-17T02:25:27.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f3274060860 0x7f3274062d10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3294155430 0x7f329414ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32940b2390 0x7f329414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.588+0000 7f329a1af700 1 --2- 172.21.15.100:0/3173677040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3294068be0 0x7f32941401b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.592+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 >> 172.21.15.100:0/3173677040 conn(0x7f329406e4e0 msgr2=0x7f3294067c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:27.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.592+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 shutdown_connections 2024-08-17T02:25:27.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.592+0000 7f329a1af700 1 -- 172.21.15.100:0/3173677040 wait complete. 2024-08-17T02:25:27.594 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 4 to deep-scrub 2024-08-17T02:25:27.603 INFO:tasks.ceph:Scrubbing osd.5 2024-08-17T02:25:27.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:25:27.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.712+0000 7f09f30b3700 1 Processor -- start 2024-08-17T02:25:27.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.712+0000 7f09f30b3700 1 -- start start 2024-08-17T02:25:27.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f30b3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09ec0cd2b0 0x7f09ec0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f30b3700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec0cc480 con 0x7f09ec0caeb0 2024-08-17T02:25:27.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f30b3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec159720 con 0x7f09ec0ceb70 2024-08-17T02:25:27.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f30b3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ec0cb870 con 0x7f09ec0cd2b0 2024-08-17T02:25:27.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09ebfff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f09ec0caeb0 0x7f09ec0ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56042/0 (socket says 172.21.15.100:56042) 2024-08-17T02:25:27.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09ebfff700 1 -- 172.21.15.100:0/2918742377 learned_addr learned my addr 172.21.15.100:0/2918742377 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:27.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f0e4f700 1 --2- 172.21.15.100:0/2918742377 >> v2:172.21.15.123:3300/0 conn(0x7f09ec0cd2b0 0x7f09ec0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744492704 0 0) 0x7f09ec0cc480 con 0x7f09ec0caeb0 2024-08-17T02:25:27.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09d4003570 con 0x7f09ec0caeb0 2024-08-17T02:25:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108400226 0 0) 0x7f09ec159720 con 0x7f09ec0ceb70 2024-08-17T02:25:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09ec0cc480 con 0x7f09ec0ceb70 2024-08-17T02:25:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2112830222 0 0) 0x7f09d4003570 con 0x7f09ec0caeb0 2024-08-17T02:25:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 >> v2:172.21.15.123:3300/0 conn(0x7f09ec0cd2b0 msgr2=0x7f09ec0cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 --2- 172.21.15.100:0/2918742377 >> v2:172.21.15.123:3300/0 conn(0x7f09ec0cd2b0 0x7f09ec0cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 >> v1:172.21.15.123:6789/0 conn(0x7f09ec0ceb70 legacy=0x7f09ec162ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0cbb50 con 0x7f09ec0caeb0 2024-08-17T02:25:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f09dc001470 con 0x7f09ec0caeb0 2024-08-17T02:25:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 >> v1:172.21.15.100:6789/0 conn(0x7f09ec0caeb0 legacy=0x7f09ec0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d4003c40 0x7f09d4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d4004810 0x7f09d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09d4005210 0x7f09d400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec0cb870 con 0x7f09d4003c40 2024-08-17T02:25:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d400c130 con 0x7f09d4004810 2024-08-17T02:25:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d400c2c0 con 0x7f09d4005210 2024-08-17T02:25:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f0e4f700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d4003c40 0x7f09d4004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09ebfff700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d4004810 0x7f09d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f1650700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09d4005210 0x7f09d400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f0e4f700 1 -- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09d4005210 msgr2=0x7f09d400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f0e4f700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09d4005210 0x7f09d400ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f0e4f700 1 -- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d4004810 msgr2=0x7f09d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f0e4f700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d4004810 0x7f09d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f1650700 1 -- 172.21.15.100:0/2918742377 reap_dead start 2024-08-17T02:25:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f0e4f700 1 -- 172.21.15.100:0/2918742377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4003570 con 0x7f09d4003c40 2024-08-17T02:25:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09ebfff700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d4004810 0x7f09d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f0e4f700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d4003c40 0x7f09d4004030 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009c10 tx=0x7f09e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2516f26176e4d44 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e000e6e0 con 0x7f09d4003c40 2024-08-17T02:25:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e000e840 con 0x7f09d4003c40 2024-08-17T02:25:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09eb7fe700 1 -- 172.21.15.100:0/2918742377 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e000ec00 con 0x7f09d4003c40 2024-08-17T02:25:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f30b3700 1 -- 172.21.15.100:0/2918742377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d4003c40 msgr2=0x7f09d4004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.716+0000 7f09f30b3700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d4003c40 0x7f09d4004030 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009c10 tx=0x7f09e00091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 -- 172.21.15.100:0/2918742377 shutdown_connections 2024-08-17T02:25:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 --2- 172.21.15.100:0/2918742377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d4003c40 0x7f09d4004030 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 -- 172.21.15.100:0/2918742377 >> 172.21.15.100:0/2918742377 conn(0x7f09ec099550 msgr2=0x7f09ec067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 -- 172.21.15.100:0/2918742377 shutdown_connections 2024-08-17T02:25:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 -- 172.21.15.100:0/2918742377 wait complete. 2024-08-17T02:25:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 Processor -- start 2024-08-17T02:25:27.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 -- start start 2024-08-17T02:25:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d4003c40 0x7f09ec1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09ec0b4be0 0x7f09ec0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09ec069120 0x7f09ec069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec069ad0 con 0x7f09ec0b4be0 2024-08-17T02:25:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec0b56f0 con 0x7f09d4003c40 2024-08-17T02:25:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f30b3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec0b0a10 con 0x7f09ec069120 2024-08-17T02:25:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09ec0b4be0 0x7f09ec0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09ec0b4be0 0x7f09ec0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48160/0 (socket says 172.21.15.100:48160) 2024-08-17T02:25:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 -- 172.21.15.100:0/4102909620 learned_addr learned my addr 172.21.15.100:0/4102909620 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09f1650700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09ec069120 0x7f09ec069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 -- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09ec069120 msgr2=0x7f09ec069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09ec069120 0x7f09ec069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 -- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d4003c40 msgr2=0x7f09ec1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d4003c40 0x7f09ec1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 -- 172.21.15.100:0/4102909620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec15a7b0 con 0x7f09ec0b4be0 2024-08-17T02:25:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.720+0000 7f09ebfff700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09ec0b4be0 0x7f09ec0b5050 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00e810 tx=0x7f09dc001530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.724+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc012590 con 0x7f09ec0b4be0 2024-08-17T02:25:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.724+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc0126f0 con 0x7f09ec0b4be0 2024-08-17T02:25:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.724+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec0b00e0 con 0x7f09ec0b4be0 2024-08-17T02:25:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.724+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc012a90 con 0x7f09ec0b4be0 2024-08-17T02:25:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.724+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec0b06a0 con 0x7f09ec0b4be0 2024-08-17T02:25:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.724+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f09dc01d460 con 0x7f09ec0b4be0 2024-08-17T02:25:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.724+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09ec0638f0 con 0x7f09ec0b4be0 2024-08-17T02:25:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.724+0000 7f09e8ff9700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f09d8060860 0x7f09d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f09dc07c720 con 0x7f09ec0b4be0 2024-08-17T02:25:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09e8ff9700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f09d8066750 0x7f09d8068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09f0e4f700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f09d8060860 0x7f09d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 --> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09d8069210 con 0x7f09d8066750 2024-08-17T02:25:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09dc07ed60 con 0x7f09ec0b4be0 2024-08-17T02:25:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09f1650700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f09d8066750 0x7f09d8068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09f0e4f700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f09d8060860 0x7f09d8062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09e00091e0 tx=0x7f09e0019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09f1650700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f09d8066750 0x7f09d8068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.728+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 <== osd.5 v2:172.21.15.123:6800/24438 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f09d8069210 con 0x7f09d8066750 2024-08-17T02:25:27.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 --> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f09ec0afe20 con 0x7f09d8066750 2024-08-17T02:25:27.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09e8ff9700 1 -- 172.21.15.100:0/4102909620 <== osd.5 v2:172.21.15.123:6800/24438 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f09ec0afe20 con 0x7f09d8066750 2024-08-17T02:25:27.751 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:25:27.752 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:25:27.752 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:25:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f09d8066750 msgr2=0x7f09d8068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6800/24438,v1:172.21.15.123:6801/24438] conn(0x7f09d8066750 0x7f09d8068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f09d8060860 msgr2=0x7f09d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f09d8060860 0x7f09d8062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09e00091e0 tx=0x7f09e0019000 comp rx=0 tx=0).stop 2024-08-17T02:25:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09ec0b4be0 msgr2=0x7f09ec0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 --2- 172.21.15.100:0/4102909620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09ec0b4be0 0x7f09ec0b5050 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00e810 tx=0x7f09dc001530 comp rx=0 tx=0).stop 2024-08-17T02:25:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f1650700 1 -- 172.21.15.100:0/4102909620 reap_dead start 2024-08-17T02:25:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 shutdown_connections 2024-08-17T02:25:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 >> 172.21.15.100:0/4102909620 conn(0x7f09ec099550 msgr2=0x7f09ec0acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.748+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 shutdown_connections 2024-08-17T02:25:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.752+0000 7f09f30b3700 1 -- 172.21.15.100:0/4102909620 wait complete. 2024-08-17T02:25:27.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-08-17T02:25:27.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d7fff700 1 Processor -- start 2024-08-17T02:25:27.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d7fff700 1 -- start start 2024-08-17T02:25:27.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d7fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe1d8068c10 0x7fe1d8158a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d7fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d80aa270 con 0x7fe1d8158fa0 2024-08-17T02:25:27.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d7fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d80aa550 con 0x7fe1d8068840 2024-08-17T02:25:27.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d7fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d8069090 con 0x7fe1d8068c10 2024-08-17T02:25:27.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d77fe700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe1d8158fa0 0x7fe1d80ab310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56052/0 (socket says 172.21.15.100:56052) 2024-08-17T02:25:27.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d77fe700 1 -- 172.21.15.100:0/3286942384 learned_addr learned my addr 172.21.15.100:0/3286942384 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:27.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d67fc700 1 --2- 172.21.15.100:0/3286942384 >> v2:172.21.15.123:3300/0 conn(0x7fe1d8068c10 0x7fe1d8158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d5ffb700 1 -- 172.21.15.100:0/3286942384 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877701189 0 0) 0x7fe1d80aa270 con 0x7fe1d8158fa0 2024-08-17T02:25:27.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d5ffb700 1 -- 172.21.15.100:0/3286942384 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c4003570 con 0x7fe1d8158fa0 2024-08-17T02:25:27.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d67fc700 1 -- 172.21.15.100:0/3286942384 >> v1:172.21.15.123:6789/0 conn(0x7fe1d8068840 legacy=0x7fe1d80a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d67fc700 1 -- 172.21.15.100:0/3286942384 >> v1:172.21.15.100:6789/0 conn(0x7fe1d8158fa0 legacy=0x7fe1d80ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d67fc700 1 -- 172.21.15.100:0/3286942384 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d80aa830 con 0x7fe1d8068c10 2024-08-17T02:25:27.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d67fc700 1 --2- 172.21.15.100:0/3286942384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8068c10 0x7fe1d8158a60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009910 tx=0x7fe1c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65aa57428de5f140 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d5ffb700 1 -- 172.21.15.100:0/3286942384 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b640 con 0x7fe1d8068c10 2024-08-17T02:25:27.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d5ffb700 1 -- 172.21.15.100:0/3286942384 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000b7a0 con 0x7fe1d8068c10 2024-08-17T02:25:27.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.868+0000 7fe1d5ffb700 1 -- 172.21.15.100:0/3286942384 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000bb20 con 0x7fe1d8068c10 2024-08-17T02:25:27.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.872+0000 7fe1d7fff700 1 -- 172.21.15.100:0/3286942384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8068c10 msgr2=0x7fe1d8158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.872+0000 7fe1d7fff700 1 --2- 172.21.15.100:0/3286942384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8068c10 0x7fe1d8158a60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009910 tx=0x7fe1c00091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.872+0000 7fe1d7fff700 1 -- 172.21.15.100:0/3286942384 shutdown_connections 2024-08-17T02:25:27.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.872+0000 7fe1d7fff700 1 --2- 172.21.15.100:0/3286942384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8068c10 0x7fe1d8158a60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.872+0000 7fe1d7fff700 1 -- 172.21.15.100:0/3286942384 >> 172.21.15.100:0/3286942384 conn(0x7fe1d806d800 msgr2=0x7fe1d806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:27.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.872+0000 7fe1d7fff700 1 -- 172.21.15.100:0/3286942384 shutdown_connections 2024-08-17T02:25:27.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.872+0000 7fe1d7fff700 1 -- 172.21.15.100:0/3286942384 wait complete. 2024-08-17T02:25:27.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d7fff700 1 Processor -- start 2024-08-17T02:25:27.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d7fff700 1 -- start start 2024-08-17T02:25:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d8068840 0x7fe1d80b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d7fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d8068c10 0x7fe1d8169d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8158fa0 0x7fe1d8147f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d7fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d816a2b0 con 0x7fe1d8068840 2024-08-17T02:25:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d7fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80b1540 con 0x7fe1d8158fa0 2024-08-17T02:25:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d7fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d81556f0 con 0x7fe1d8068c10 2024-08-17T02:25:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d8068840 0x7fe1d80b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d8068840 0x7fe1d80b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48162/0 (socket says 172.21.15.100:48162) 2024-08-17T02:25:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 -- 172.21.15.100:0/2215272760 learned_addr learned my addr 172.21.15.100:0/2215272760 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d77fe700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8158fa0 0x7fe1d8147f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d67fc700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d8068c10 0x7fe1d8169d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 -- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d8068c10 msgr2=0x7fe1d8169d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d8068c10 0x7fe1d8169d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 -- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8158fa0 msgr2=0x7fe1d8147f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:27.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8158fa0 0x7fe1d8147f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:27.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 -- 172.21.15.100:0/2215272760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c0009420 con 0x7fe1d8068840 2024-08-17T02:25:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.876+0000 7fe1d6ffd700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d8068840 0x7fe1d80b19d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800c590 tx=0x7fe1c800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.880+0000 7fe1ceffd700 1 -- 172.21.15.100:0/2215272760 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8011590 con 0x7fe1d8068840 2024-08-17T02:25:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.880+0000 7fe1ceffd700 1 -- 172.21.15.100:0/2215272760 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c8011710 con 0x7fe1d8068840 2024-08-17T02:25:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.880+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d80b0930 con 0x7fe1d8068840 2024-08-17T02:25:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.880+0000 7fe1ceffd700 1 -- 172.21.15.100:0/2215272760 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8011ad0 con 0x7fe1d8068840 2024-08-17T02:25:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.880+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d80b0ef0 con 0x7fe1d8068840 2024-08-17T02:25:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.880+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d80638f0 con 0x7fe1d8068840 2024-08-17T02:25:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.880+0000 7fe1ceffd700 1 -- 172.21.15.100:0/2215272760 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe1c801b3b0 con 0x7fe1d8068840 2024-08-17T02:25:27.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.880+0000 7fe1ceffd700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1b8060860 0x7fe1b8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:27.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.884+0000 7fe1d67fc700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1b8060860 0x7fe1b8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:27.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.884+0000 7fe1ceffd700 1 -- 172.21.15.100:0/2215272760 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe1c807bd90 con 0x7fe1d8068840 2024-08-17T02:25:27.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.884+0000 7fe1d67fc700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1b8060860 0x7fe1b8062d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00091e0 tx=0x7fe1c0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:27.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:27.892+0000 7fe1ceffd700 1 -- 172.21.15.100:0/2215272760 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe1c804c480 con 0x7fe1d8068840 2024-08-17T02:25:28.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.052+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1d80b1680 con 0x7fe1b8060860 2024-08-17T02:25:28.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.052+0000 7fe1ceffd700 1 -- 172.21.15.100:0/2215272760 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe1c0020040 con 0x7fe1b8060860 2024-08-17T02:25:28.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1b8060860 msgr2=0x7fe1b8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1b8060860 0x7fe1b8062d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00091e0 tx=0x7fe1c0017000 comp rx=0 tx=0).stop 2024-08-17T02:25:28.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d8068840 msgr2=0x7fe1d80b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d8068840 0x7fe1d80b19d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800c590 tx=0x7fe1c800c8a0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 shutdown_connections 2024-08-17T02:25:28.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fe1b8060860 0x7fe1b8062d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8158fa0 0x7fe1d8147f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d8068c10 0x7fe1d8169d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 --2- 172.21.15.100:0/2215272760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d8068840 0x7fe1d80b19d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 >> 172.21.15.100:0/2215272760 conn(0x7fe1d806d800 msgr2=0x7fe1d81572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 shutdown_connections 2024-08-17T02:25:28.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.056+0000 7fe1d7fff700 1 -- 172.21.15.100:0/2215272760 wait complete. 2024-08-17T02:25:28.060 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 5 to deep-scrub 2024-08-17T02:25:28.069 INFO:tasks.ceph:Scrubbing osd.6 2024-08-17T02:25:28.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:25:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.176+0000 7f063c960700 1 Processor -- start 2024-08-17T02:25:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.176+0000 7f063c960700 1 -- start start 2024-08-17T02:25:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f06380b1500 0x7f06380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0638068ce0 0x7f06380aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06380aeb20 con 0x7f0638142960 2024-08-17T02:25:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06380694b0 con 0x7f06380b1500 2024-08-17T02:25:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0638069790 con 0x7f0638068ce0 2024-08-17T02:25:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063659c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f06380b1500 0x7f06380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f0636d9d700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0638142960 0x7f06380b0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56060/0 (socket says 172.21.15.100:56060) 2024-08-17T02:25:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f0636d9d700 1 -- 172.21.15.100:0/4039483704 learned_addr learned my addr 172.21.15.100:0/4039483704 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063759e700 1 --2- 172.21.15.100:0/4039483704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0638068ce0 0x7f06380aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f0635d9b700 1 -- 172.21.15.100:0/4039483704 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427901654 0 0) 0x7f06380aeb20 con 0x7f0638142960 2024-08-17T02:25:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f0635d9b700 1 -- 172.21.15.100:0/4039483704 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0620003570 con 0x7f0638142960 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063659c700 1 -- 172.21.15.100:0/4039483704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0638068ce0 msgr2=0x7f06380aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063659c700 1 --2- 172.21.15.100:0/4039483704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0638068ce0 0x7f06380aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063659c700 1 -- 172.21.15.100:0/4039483704 >> v1:172.21.15.100:6789/0 conn(0x7f0638142960 legacy=0x7f06380b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063659c700 1 -- 172.21.15.100:0/4039483704 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638069a70 con 0x7f06380b1500 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063659c700 1 --2- 172.21.15.100:0/4039483704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06380b1500 0x7f06380687a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f06280098e0 tx=0x7f06280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0eff0214d71c54 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f0635d9b700 1 -- 172.21.15.100:0/4039483704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628011630 con 0x7f06380b1500 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f0635d9b700 1 -- 172.21.15.100:0/4039483704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0628011790 con 0x7f06380b1500 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f0635d9b700 1 -- 172.21.15.100:0/4039483704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628011b30 con 0x7f06380b1500 2024-08-17T02:25:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 -- 172.21.15.100:0/4039483704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06380b1500 msgr2=0x7f06380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 --2- 172.21.15.100:0/4039483704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06380b1500 0x7f06380687a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f06280098e0 tx=0x7f06280091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 -- 172.21.15.100:0/4039483704 shutdown_connections 2024-08-17T02:25:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 --2- 172.21.15.100:0/4039483704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0638068ce0 0x7f06380aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 --2- 172.21.15.100:0/4039483704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06380b1500 0x7f06380687a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 -- 172.21.15.100:0/4039483704 >> 172.21.15.100:0/4039483704 conn(0x7f063806cd20 msgr2=0x7f063806d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 -- 172.21.15.100:0/4039483704 shutdown_connections 2024-08-17T02:25:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.180+0000 7f063c960700 1 -- 172.21.15.100:0/4039483704 wait complete. 2024-08-17T02:25:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 Processor -- start 2024-08-17T02:25:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 -- start start 2024-08-17T02:25:28.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0638068ce0 0x7f06380c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06380b1500 0x7f0638158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0638142960 0x7f063815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06380cb930 con 0x7f0638068ce0 2024-08-17T02:25:28.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063816d0f0 con 0x7f0638142960 2024-08-17T02:25:28.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063816d3d0 con 0x7f06380b1500 2024-08-17T02:25:28.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0638068ce0 0x7f06380c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0638068ce0 0x7f06380c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48190/0 (socket says 172.21.15.100:48190) 2024-08-17T02:25:28.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 -- 172.21.15.100:0/3377607546 learned_addr learned my addr 172.21.15.100:0/3377607546 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:28.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063659c700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06380b1500 0x7f0638158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063759e700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0638142960 0x7f063815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 -- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06380b1500 msgr2=0x7f0638158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06380b1500 0x7f0638158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 -- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0638142960 msgr2=0x7f063815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0638142960 0x7f063815dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 -- 172.21.15.100:0/3377607546 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0628009420 con 0x7f0638068ce0 2024-08-17T02:25:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063759e700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0638142960 0x7f063815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f0636d9d700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0638068ce0 0x7f06380c7180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c590 tx=0x7f062c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c005cd0 con 0x7f0638068ce0 2024-08-17T02:25:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c005e50 con 0x7f0638068ce0 2024-08-17T02:25:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063816d6b0 con 0x7f0638068ce0 2024-08-17T02:25:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c007320 con 0x7f0638068ce0 2024-08-17T02:25:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063816cd10 con 0x7f0638068ce0 2024-08-17T02:25:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.184+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f06380644b0 con 0x7f0638068ce0 2024-08-17T02:25:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f062c01b3b0 con 0x7f0638068ce0 2024-08-17T02:25:28.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f061effd700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0624060860 0x7f0624062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f062c07c790 con 0x7f0638068ce0 2024-08-17T02:25:28.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f063659c700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0624060860 0x7f0624062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f061effd700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7f0624066750 0x7f0624068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 --> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0624069210 con 0x7f0624066750 2024-08-17T02:25:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062c07ca80 con 0x7f0638068ce0 2024-08-17T02:25:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f063759e700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7f0624066750 0x7f0624068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f063659c700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0624060860 0x7f0624062d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06280098e0 tx=0x7f0628017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.188+0000 7f063759e700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7f0624066750 0x7f0624068b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.192+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 <== osd.6 v2:172.21.15.123:6816/24440 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0624069210 con 0x7f0624066750 2024-08-17T02:25:28.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.208+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 --> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f06380b1930 con 0x7f0624066750 2024-08-17T02:25:28.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f061effd700 1 -- 172.21.15.100:0/3377607546 <== osd.6 v2:172.21.15.123:6816/24440 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f06380b1930 con 0x7f0624066750 2024-08-17T02:25:28.212 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:25:28.213 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:25:28.213 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:25:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7f0624066750 msgr2=0x7f0624068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6816/24440,v1:172.21.15.123:6817/24440] conn(0x7f0624066750 0x7f0624068b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0624060860 msgr2=0x7f0624062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f0624060860 0x7f0624062d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06280098e0 tx=0x7f0628017000 comp rx=0 tx=0).stop 2024-08-17T02:25:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0638068ce0 msgr2=0x7f06380c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 --2- 172.21.15.100:0/3377607546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0638068ce0 0x7f06380c7180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c590 tx=0x7f062c007e00 comp rx=0 tx=0).stop 2024-08-17T02:25:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063759e700 1 -- 172.21.15.100:0/3377607546 reap_dead start 2024-08-17T02:25:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 shutdown_connections 2024-08-17T02:25:28.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 >> 172.21.15.100:0/3377607546 conn(0x7f063806cd20 msgr2=0x7f063813d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 shutdown_connections 2024-08-17T02:25:28.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.212+0000 7f063c960700 1 -- 172.21.15.100:0/3377607546 wait complete. 2024-08-17T02:25:28.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-08-17T02:25:28.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.332+0000 7f293ffff700 1 Processor -- start 2024-08-17T02:25:28.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.332+0000 7f293ffff700 1 -- start start 2024-08-17T02:25:28.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.332+0000 7f293ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29400cb560 0x7f2940161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.332+0000 7f293ffff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29400cbaa0 con 0x7f29400b14a0 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.332+0000 7f293ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2940157630 con 0x7f2940154900 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.332+0000 7f293ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2940068790 con 0x7f29400cb560 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.332+0000 7f293e7fc700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f29400b14a0 0x7f29400cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56066/0 (socket says 172.21.15.100:56066) 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.332+0000 7f293e7fc700 1 -- 172.21.15.100:0/1044250107 learned_addr learned my addr 172.21.15.100:0/1044250107 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293f7fe700 1 --2- 172.21.15.100:0/1044250107 >> v2:172.21.15.123:3300/0 conn(0x7f29400cb560 0x7f2940161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293dffb700 1 -- 172.21.15.100:0/1044250107 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815771964 0 0) 0x7f29400cbaa0 con 0x7f29400b14a0 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293dffb700 1 -- 172.21.15.100:0/1044250107 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2920003570 con 0x7f29400b14a0 2024-08-17T02:25:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293f7fe700 1 -- 172.21.15.100:0/1044250107 >> v1:172.21.15.123:6789/0 conn(0x7f2940154900 legacy=0x7f29400b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293f7fe700 1 -- 172.21.15.100:0/1044250107 >> v1:172.21.15.100:6789/0 conn(0x7f29400b14a0 legacy=0x7f29400cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293f7fe700 1 -- 172.21.15.100:0/1044250107 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2940157910 con 0x7f29400cb560 2024-08-17T02:25:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293f7fe700 1 --2- 172.21.15.100:0/1044250107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 0x7f2940161ad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2938009e60 tx=0x7f2938009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf3ff985a57342 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293dffb700 1 -- 172.21.15.100:0/1044250107 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293800e070 con 0x7f29400cb560 2024-08-17T02:25:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293dffb700 1 -- 172.21.15.100:0/1044250107 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293800bb60 con 0x7f29400cb560 2024-08-17T02:25:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293dffb700 1 -- 172.21.15.100:0/1044250107 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938012430 con 0x7f29400cb560 2024-08-17T02:25:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 -- 172.21.15.100:0/1044250107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 msgr2=0x7f2940161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 --2- 172.21.15.100:0/1044250107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 0x7f2940161ad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2938009e60 tx=0x7f2938009780 comp rx=0 tx=0).stop 2024-08-17T02:25:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 -- 172.21.15.100:0/1044250107 shutdown_connections 2024-08-17T02:25:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 --2- 172.21.15.100:0/1044250107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 0x7f2940161ad0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 -- 172.21.15.100:0/1044250107 >> 172.21.15.100:0/1044250107 conn(0x7f29400678d0 msgr2=0x7f2940067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 -- 172.21.15.100:0/1044250107 shutdown_connections 2024-08-17T02:25:28.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 -- 172.21.15.100:0/1044250107 wait complete. 2024-08-17T02:25:28.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 Processor -- start 2024-08-17T02:25:28.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.336+0000 7f293ffff700 1 -- start start 2024-08-17T02:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29400b14a0 0x7f294016d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 0x7f2940158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2940154900 0x7f29401505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29400b0bd0 con 0x7f2940154900 2024-08-17T02:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29400a9990 con 0x7f29400cb560 2024-08-17T02:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29400a9c70 con 0x7f29400b14a0 2024-08-17T02:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2940154900 0x7f29401505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2940154900 0x7f29401505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48204/0 (socket says 172.21.15.100:48204) 2024-08-17T02:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 -- 172.21.15.100:0/564134651 learned_addr learned my addr 172.21.15.100:0/564134651 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293effd700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29400b14a0 0x7f294016d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293e7fc700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 0x7f2940158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 -- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29400b14a0 msgr2=0x7f294016d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29400b14a0 0x7f294016d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 -- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 msgr2=0x7f2940158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 0x7f2940158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 -- 172.21.15.100:0/564134651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938009a20 con 0x7f2940154900 2024-08-17T02:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293f7fe700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2940154900 0x7f29401505a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f294006c2d0 tx=0x7f2938019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f2936ffd700 1 -- 172.21.15.100:0/564134651 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938027470 con 0x7f2940154900 2024-08-17T02:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f2936ffd700 1 -- 172.21.15.100:0/564134651 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29380275f0 con 0x7f2940154900 2024-08-17T02:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29400a9f50 con 0x7f2940154900 2024-08-17T02:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f2936ffd700 1 -- 172.21.15.100:0/564134651 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29380279b0 con 0x7f2940154900 2024-08-17T02:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29400aa2e0 con 0x7f2940154900 2024-08-17T02:25:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29400aa470 con 0x7f2940154900 2024-08-17T02:25:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.340+0000 7f2936ffd700 1 -- 172.21.15.100:0/564134651 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2938024070 con 0x7f2940154900 2024-08-17T02:25:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.344+0000 7f2936ffd700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2924060860 0x7f2924062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.344+0000 7f2936ffd700 1 -- 172.21.15.100:0/564134651 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f293807f330 con 0x7f2940154900 2024-08-17T02:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.344+0000 7f293effd700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2924060860 0x7f2924062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.344+0000 7f293effd700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2924060860 0x7f2924062d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f29300039e0 tx=0x7f2930007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.348+0000 7f2936ffd700 1 -- 172.21.15.100:0/564134651 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f293804f980 con 0x7f2940154900 2024-08-17T02:25:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f294016d8d0 con 0x7f2924060860 2024-08-17T02:25:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f2936ffd700 1 -- 172.21.15.100:0/564134651 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f294016d8d0 con 0x7f2924060860 2024-08-17T02:25:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2924060860 msgr2=0x7f2924062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2924060860 0x7f2924062d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f29300039e0 tx=0x7f2930007ec0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2940154900 msgr2=0x7f29401505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2940154900 0x7f29401505a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f294006c2d0 tx=0x7f2938019000 comp rx=0 tx=0).stop 2024-08-17T02:25:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 shutdown_connections 2024-08-17T02:25:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f2924060860 0x7f2924062d10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2940154900 0x7f29401505a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29400cb560 0x7f2940158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 --2- 172.21.15.100:0/564134651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29400b14a0 0x7f294016d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 >> 172.21.15.100:0/564134651 conn(0x7f29400678d0 msgr2=0x7f2940153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 shutdown_connections 2024-08-17T02:25:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.508+0000 7f293ffff700 1 -- 172.21.15.100:0/564134651 wait complete. 2024-08-17T02:25:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 6 to deep-scrub 2024-08-17T02:25:28.523 INFO:tasks.ceph:Scrubbing osd.7 2024-08-17T02:25:28.524 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:25:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.632+0000 7f5b616cc700 1 Processor -- start 2024-08-17T02:25:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.632+0000 7f5b616cc700 1 -- start start 2024-08-17T02:25:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5b5c13f7f0 0x7f5b5c0ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5c0b0660 con 0x7f5b5c06f1d0 2024-08-17T02:25:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5c0afd30 con 0x7f5b5c13fbc0 2024-08-17T02:25:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c0abdc0 con 0x7f5b5c13f7f0 2024-08-17T02:25:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b60ecb700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5b5c06f1d0 0x7f5b5c156700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56082/0 (socket says 172.21.15.100:56082) 2024-08-17T02:25:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b60ecb700 1 -- 172.21.15.100:0/1020816960 learned_addr learned my addr 172.21.15.100:0/1020816960 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5bfff700 1 --2- 172.21.15.100:0/1020816960 >> v2:172.21.15.123:3300/0 conn(0x7f5b5c13f7f0 0x7f5b5c0ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652706157 0 0) 0x7f5b5c0b0660 con 0x7f5b5c06f1d0 2024-08-17T02:25:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b48003570 con 0x7f5b5c06f1d0 2024-08-17T02:25:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943171672 0 0) 0x7f5b5c0afd30 con 0x7f5b5c13fbc0 2024-08-17T02:25:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b5c0b0660 con 0x7f5b5c13fbc0 2024-08-17T02:25:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 731593981 0 0) 0x7f5b48003570 con 0x7f5b5c06f1d0 2024-08-17T02:25:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 >> v2:172.21.15.123:3300/0 conn(0x7f5b5c13f7f0 msgr2=0x7f5b5c0ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 --2- 172.21.15.100:0/1020816960 >> v2:172.21.15.123:3300/0 conn(0x7f5b5c13f7f0 0x7f5b5c0ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 >> v1:172.21.15.123:6789/0 conn(0x7f5b5c13fbc0 legacy=0x7f5b5c06eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0b0010 con 0x7f5b5c06f1d0 2024-08-17T02:25:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f5b540017f0 con 0x7f5b5c06f1d0 2024-08-17T02:25:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 >> v1:172.21.15.100:6789/0 conn(0x7f5b5c06f1d0 legacy=0x7f5b5c156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b48003bf0 0x7f5b48003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b48004810 0x7f5b48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b480051c0 0x7f5b4800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0abdc0 con 0x7f5b48003bf0 2024-08-17T02:25:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4800c160 con 0x7f5b480051c0 2024-08-17T02:25:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4800c320 con 0x7f5b48004810 2024-08-17T02:25:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5bfff700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b48003bf0 0x7f5b48003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5b7fe700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b48004810 0x7f5b48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b60ecb700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b480051c0 0x7f5b4800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5bfff700 1 -- 172.21.15.100:0/1020816960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b48004810 msgr2=0x7f5b48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5bfff700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b48004810 0x7f5b48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5bfff700 1 -- 172.21.15.100:0/1020816960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b480051c0 msgr2=0x7f5b4800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5bfff700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b480051c0 0x7f5b4800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5bfff700 1 -- 172.21.15.100:0/1020816960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b48003570 con 0x7f5b48003bf0 2024-08-17T02:25:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b60ecb700 1 -- 172.21.15.100:0/1020816960 reap_dead start 2024-08-17T02:25:28.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5bfff700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b48003bf0 0x7f5b48003fe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006f00 tx=0x7f5b440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dafe2a775e8503 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4400e710 con 0x7f5b48003bf0 2024-08-17T02:25:28.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4400e870 con 0x7f5b48003bf0 2024-08-17T02:25:28.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b5affd700 1 -- 172.21.15.100:0/1020816960 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4400ec30 con 0x7f5b48003bf0 2024-08-17T02:25:28.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 -- 172.21.15.100:0/1020816960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b48003bf0 msgr2=0x7f5b48003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b48003bf0 0x7f5b48003fe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006f00 tx=0x7f5b440091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 -- 172.21.15.100:0/1020816960 shutdown_connections 2024-08-17T02:25:28.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 --2- 172.21.15.100:0/1020816960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b48003bf0 0x7f5b48003fe0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.636+0000 7f5b616cc700 1 -- 172.21.15.100:0/1020816960 >> 172.21.15.100:0/1020816960 conn(0x7f5b5c066e00 msgr2=0x7f5b5c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.640+0000 7f5b616cc700 1 -- 172.21.15.100:0/1020816960 shutdown_connections 2024-08-17T02:25:28.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.640+0000 7f5b616cc700 1 -- 172.21.15.100:0/1020816960 wait complete. 2024-08-17T02:25:28.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.640+0000 7f5b616cc700 1 Processor -- start 2024-08-17T02:25:28.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.640+0000 7f5b616cc700 1 -- start start 2024-08-17T02:25:28.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b616cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b48003bf0 0x7f5b5c0aa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b616cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0aa8a0 0x7f5b5c0a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b616cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b5c0a65c0 0x7f5b5c0a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b616cc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c157370 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b616cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0aad10 con 0x7f5b48003bf0 2024-08-17T02:25:28.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b616cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0a7dc0 con 0x7f5b5c0a65c0 2024-08-17T02:25:28.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0aa8a0 0x7f5b5c0a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b48003bf0 0x7f5b5c0aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0aa8a0 0x7f5b5c0a6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48226/0 (socket says 172.21.15.100:48226) 2024-08-17T02:25:28.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b48003bf0 0x7f5b5c0aa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.100:33666/0 (socket says 172.21.15.100:33666) 2024-08-17T02:25:28.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 -- 172.21.15.100:0/873813520 learned_addr learned my addr 172.21.15.100:0/873813520 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:28.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b60ecb700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b5c0a65c0 0x7f5b5c0a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 -- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b5c0a65c0 msgr2=0x7f5b5c0a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b5c0a65c0 0x7f5b5c0a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 -- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b48003bf0 msgr2=0x7f5b5c0aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b48003bf0 0x7f5b5c0aa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 -- 172.21.15.100:0/873813520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0a80a0 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b60ecb700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b5c0a65c0 0x7f5b5c0a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b5b7fe700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0aa8a0 0x7f5b5c0a6080 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c007a80 tx=0x7f5b4c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00f8d0 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c00fa30 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c0a8380 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00fdf0 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.644+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c0a8940 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b5c0644b0 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5b4c013600 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b53fff700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5b3c060910 0x7f5b3c062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f5b4c07d070 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b53fff700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f5b3c066800 0x7f5b3c068c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 --> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b3c0692c0 con 0x7f5b3c066800 2024-08-17T02:25:28.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b4c07f4f0 con 0x7f5b5c0aa8a0 2024-08-17T02:25:28.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b5bfff700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5b3c060910 0x7f5b3c062dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b60ecb700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f5b3c066800 0x7f5b3c068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b5bfff700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5b3c060910 0x7f5b3c062dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b440091e0 tx=0x7f5b4401a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.648+0000 7f5b60ecb700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f5b3c066800 0x7f5b3c068c10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.652+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 <== osd.7 v2:172.21.15.123:6824/24441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5b3c0692c0 con 0x7f5b3c066800 2024-08-17T02:25:28.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 --> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5b5c06ed70 con 0x7f5b3c066800 2024-08-17T02:25:28.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b53fff700 1 -- 172.21.15.100:0/873813520 <== osd.7 v2:172.21.15.123:6824/24441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5b5c06ed70 con 0x7f5b3c066800 2024-08-17T02:25:28.670 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-08-17T02:25:28.670 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:25:28.671 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-08-17T02:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f5b3c066800 msgr2=0x7f5b3c068c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6824/24441,v1:172.21.15.123:6825/24441] conn(0x7f5b3c066800 0x7f5b3c068c10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5b3c060910 msgr2=0x7f5b3c062dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f5b3c060910 0x7f5b3c062dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b440091e0 tx=0x7f5b4401a000 comp rx=0 tx=0).stop 2024-08-17T02:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0aa8a0 msgr2=0x7f5b5c0a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 --2- 172.21.15.100:0/873813520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0aa8a0 0x7f5b5c0a6080 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c007a80 tx=0x7f5b4c007d30 comp rx=0 tx=0).stop 2024-08-17T02:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b60ecb700 1 -- 172.21.15.100:0/873813520 reap_dead start 2024-08-17T02:25:28.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 shutdown_connections 2024-08-17T02:25:28.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 >> 172.21.15.100:0/873813520 conn(0x7f5b5c066e00 msgr2=0x7f5b5c069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 shutdown_connections 2024-08-17T02:25:28.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.668+0000 7f5b616cc700 1 -- 172.21.15.100:0/873813520 wait complete. 2024-08-17T02:25:28.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-08-17T02:25:28.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967e5de700 1 Processor -- start 2024-08-17T02:25:28.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967e5de700 1 -- start start 2024-08-17T02:25:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967e5de700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96780af0a0 0x7f96780a9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967e5de700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96780b0700 0x7f96780b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967e5de700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96780b0270 con 0x7f96780a9180 2024-08-17T02:25:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967e5de700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96780af660 con 0x7f96780af0a0 2024-08-17T02:25:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967e5de700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96780af940 con 0x7f96780b0700 2024-08-17T02:25:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967cddb700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96780af0a0 0x7f96780a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967cddb700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96780af0a0 0x7f96780a9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48238/0 (socket says 172.21.15.100:48238) 2024-08-17T02:25:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967cddb700 1 -- 172.21.15.100:0/3713178884 learned_addr learned my addr 172.21.15.100:0/3713178884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967dddd700 1 --2- 172.21.15.100:0/3713178884 >> v2:172.21.15.123:3300/0 conn(0x7f96780b0700 0x7f96780b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967cddb700 1 -- 172.21.15.100:0/3713178884 >> v2:172.21.15.123:3300/0 conn(0x7f96780b0700 msgr2=0x7f96780b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967cddb700 1 --2- 172.21.15.100:0/3713178884 >> v2:172.21.15.123:3300/0 conn(0x7f96780b0700 0x7f96780b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967cddb700 1 -- 172.21.15.100:0/3713178884 >> v1:172.21.15.123:6789/0 conn(0x7f96780a9180 legacy=0x7f9678155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.792+0000 7f967cddb700 1 -- 172.21.15.100:0/3713178884 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96780afc20 con 0x7f96780af0a0 2024-08-17T02:25:28.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967cddb700 1 --2- 172.21.15.100:0/3713178884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780af0a0 0x7f96780a9550 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f96780aa7e0 tx=0x7f9668009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe38d95013d9e1c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f9677fff700 1 -- 172.21.15.100:0/3713178884 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966800e070 con 0x7f96780af0a0 2024-08-17T02:25:28.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f9677fff700 1 -- 172.21.15.100:0/3713178884 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966800ba20 con 0x7f96780af0a0 2024-08-17T02:25:28.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f9677fff700 1 -- 172.21.15.100:0/3713178884 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680092e0 con 0x7f96780af0a0 2024-08-17T02:25:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 -- 172.21.15.100:0/3713178884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780af0a0 msgr2=0x7f96780a9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 --2- 172.21.15.100:0/3713178884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780af0a0 0x7f96780a9550 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f96780aa7e0 tx=0x7f9668009840 comp rx=0 tx=0).stop 2024-08-17T02:25:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 -- 172.21.15.100:0/3713178884 shutdown_connections 2024-08-17T02:25:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 --2- 172.21.15.100:0/3713178884 >> v2:172.21.15.123:3300/0 conn(0x7f96780b0700 0x7f96780b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 --2- 172.21.15.100:0/3713178884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780af0a0 0x7f96780a9550 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 -- 172.21.15.100:0/3713178884 >> 172.21.15.100:0/3713178884 conn(0x7f9678066e00 msgr2=0x7f9678067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 -- 172.21.15.100:0/3713178884 shutdown_connections 2024-08-17T02:25:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 -- 172.21.15.100:0/3713178884 wait complete. 2024-08-17T02:25:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 Processor -- start 2024-08-17T02:25:28.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.796+0000 7f967e5de700 1 -- start start 2024-08-17T02:25:28.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96780a9180 0x7f9678099620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96780af0a0 0x7f96780cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780b0700 0x7f9678097da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96780982e0 con 0x7f96780b0700 2024-08-17T02:25:28.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9678098580 con 0x7f96780af0a0 2024-08-17T02:25:28.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9678098860 con 0x7f96780a9180 2024-08-17T02:25:28.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967dddd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780b0700 0x7f9678097da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967d5dc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96780a9180 0x7f9678099620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967dddd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780b0700 0x7f9678097da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48254/0 (socket says 172.21.15.100:48254) 2024-08-17T02:25:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967cddb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96780af0a0 0x7f96780cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967dddd700 1 -- 172.21.15.100:0/2126422470 learned_addr learned my addr 172.21.15.100:0/2126422470 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967cddb700 1 -- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96780a9180 msgr2=0x7f9678099620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967cddb700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96780a9180 0x7f9678099620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967cddb700 1 -- 172.21.15.100:0/2126422470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780b0700 msgr2=0x7f9678097da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967cddb700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780b0700 0x7f9678097da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967cddb700 1 -- 172.21.15.100:0/2126422470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9668009ae0 con 0x7f96780af0a0 2024-08-17T02:25:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967cddb700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96780af0a0 0x7f96780cae20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9668015000 tx=0x7f9668018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f96757fa700 1 -- 172.21.15.100:0/2126422470 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680078f0 con 0x7f96780af0a0 2024-08-17T02:25:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f96757fa700 1 -- 172.21.15.100:0/2126422470 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668007a70 con 0x7f96780af0a0 2024-08-17T02:25:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9678098b40 con 0x7f96780af0a0 2024-08-17T02:25:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f96757fa700 1 -- 172.21.15.100:0/2126422470 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668007e30 con 0x7f96780af0a0 2024-08-17T02:25:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9678099100 con 0x7f96780af0a0 2024-08-17T02:25:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.800+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9678064330 con 0x7f96780af0a0 2024-08-17T02:25:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.804+0000 7f96757fa700 1 -- 172.21.15.100:0/2126422470 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9668003040 con 0x7f96780af0a0 2024-08-17T02:25:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.804+0000 7f96757fa700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9664060860 0x7f9664062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.804+0000 7f96757fa700 1 -- 172.21.15.100:0/2126422470 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f966807e560 con 0x7f96780af0a0 2024-08-17T02:25:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.804+0000 7f967d5dc700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9664060860 0x7f9664062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.804+0000 7f967d5dc700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9664060860 0x7f9664062d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f966c003a80 tx=0x7f966c007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.812+0000 7f96757fa700 1 -- 172.21.15.100:0/2126422470 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f966804ebb0 con 0x7f96780af0a0 2024-08-17T02:25:28.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.976+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f96780aee80 con 0x7f9664060860 2024-08-17T02:25:28.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.976+0000 7f96757fa700 1 -- 172.21.15.100:0/2126422470 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f96780aee80 con 0x7f9664060860 2024-08-17T02:25:28.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9664060860 msgr2=0x7f9664062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9664060860 0x7f9664062d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f966c003a80 tx=0x7f966c007ec0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96780af0a0 msgr2=0x7f96780cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:28.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96780af0a0 0x7f96780cae20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9668015000 tx=0x7f9668018000 comp rx=0 tx=0).stop 2024-08-17T02:25:28.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 shutdown_connections 2024-08-17T02:25:28.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f9664060860 0x7f9664062d10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96780b0700 0x7f9678097da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96780af0a0 0x7f96780cae20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 --2- 172.21.15.100:0/2126422470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96780a9180 0x7f9678099620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:28.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 >> 172.21.15.100:0/2126422470 conn(0x7f9678066e00 msgr2=0x7f967806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:28.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 shutdown_connections 2024-08-17T02:25:28.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:28.980+0000 7f967e5de700 1 -- 172.21.15.100:0/2126422470 wait complete. 2024-08-17T02:25:28.984 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 7 to deep-scrub 2024-08-17T02:25:28.993 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:25:29.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e13fff700 1 Processor -- start 2024-08-17T02:25:29.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e13fff700 1 -- start start 2024-08-17T02:25:29.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e13fff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e140b0320 0x7f8e1415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:29.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e13fff700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140c86e0 con 0x7f8e140ad1b0 2024-08-17T02:25:29.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e13fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140cb850 con 0x7f8e1415c620 2024-08-17T02:25:29.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e13fff700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1415d490 con 0x7f8e140b0320 2024-08-17T02:25:29.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e137fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e140b0320 0x7f8e1415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:29.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e127fc700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8e140ad1b0 0x7f8e140c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56092/0 (socket says 172.21.15.100:56092) 2024-08-17T02:25:29.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e127fc700 1 -- 172.21.15.100:0/577044920 learned_addr learned my addr 172.21.15.100:0/577044920 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:29.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e137fe700 1 -- 172.21.15.100:0/577044920 >> v1:172.21.15.123:6789/0 conn(0x7f8e1415c620 legacy=0x7f8e140acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:29.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e11ffb700 1 -- 172.21.15.100:0/577044920 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741255667 0 0) 0x7f8e140cb850 con 0x7f8e1415c620 2024-08-17T02:25:29.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.092+0000 7f8e137fe700 1 -- 172.21.15.100:0/577044920 >> v1:172.21.15.100:6789/0 conn(0x7f8e140ad1b0 legacy=0x7f8e140c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:29.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e137fe700 1 -- 172.21.15.100:0/577044920 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1415d770 con 0x7f8e140b0320 2024-08-17T02:25:29.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e137fe700 1 --2- 172.21.15.100:0/577044920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e140b0320 0x7f8e1415cd90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0098e0 tx=0x7f8e0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc42b8ea9019221 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:29.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e11ffb700 1 -- 172.21.15.100:0/577044920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c009e80 con 0x7f8e140b0320 2024-08-17T02:25:29.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e11ffb700 1 -- 172.21.15.100:0/577044920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c00b760 con 0x7f8e140b0320 2024-08-17T02:25:29.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e11ffb700 1 -- 172.21.15.100:0/577044920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00bae0 con 0x7f8e140b0320 2024-08-17T02:25:29.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 -- 172.21.15.100:0/577044920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e140b0320 msgr2=0x7f8e1415cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:29.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 --2- 172.21.15.100:0/577044920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e140b0320 0x7f8e1415cd90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0098e0 tx=0x7f8e0c0091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:29.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 -- 172.21.15.100:0/577044920 shutdown_connections 2024-08-17T02:25:29.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 --2- 172.21.15.100:0/577044920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e140b0320 0x7f8e1415cd90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:29.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 -- 172.21.15.100:0/577044920 >> 172.21.15.100:0/577044920 conn(0x7f8e140ca900 msgr2=0x7f8e140cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:29.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 -- 172.21.15.100:0/577044920 shutdown_connections 2024-08-17T02:25:29.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 -- 172.21.15.100:0/577044920 wait complete. 2024-08-17T02:25:29.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 Processor -- start 2024-08-17T02:25:29.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.096+0000 7f8e13fff700 1 -- start start 2024-08-17T02:25:29.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e140ad1b0 0x7f8e1415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:29.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e140b0320 0x7f8e140c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:29.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1415c620 0x7f8e1416a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:29.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140689b0 con 0x7f8e1415c620 2024-08-17T02:25:29.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1415afa0 con 0x7f8e140ad1b0 2024-08-17T02:25:29.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140c5630 con 0x7f8e140b0320 2024-08-17T02:25:29.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1415c620 0x7f8e1416a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:29.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1415c620 0x7f8e1416a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48278/0 (socket says 172.21.15.100:48278) 2024-08-17T02:25:29.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 -- 172.21.15.100:0/4170417233 learned_addr learned my addr 172.21.15.100:0/4170417233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:29.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e12ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e140ad1b0 0x7f8e1415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:29.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e127fc700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e140b0320 0x7f8e140c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:29.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 -- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e140b0320 msgr2=0x7f8e140c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:29.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e140b0320 0x7f8e140c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:29.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 -- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e140ad1b0 msgr2=0x7f8e1415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:29.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e140ad1b0 0x7f8e1415aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:29.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 -- 172.21.15.100:0/4170417233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c009420 con 0x7f8e1415c620 2024-08-17T02:25:29.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e12ffd700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e140ad1b0 0x7f8e1415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:25:29.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e137fe700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1415c620 0x7f8e1416a830 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0093f0 tx=0x7f8e0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:29.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e0affd700 1 -- 172.21.15.100:0/4170417233 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c01c460 con 0x7f8e1415c620 2024-08-17T02:25:29.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1416a260 con 0x7f8e1415c620 2024-08-17T02:25:29.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e0affd700 1 -- 172.21.15.100:0/4170417233 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c01c5c0 con 0x7f8e1415c620 2024-08-17T02:25:29.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e140c52c0 con 0x7f8e1415c620 2024-08-17T02:25:29.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e0affd700 1 -- 172.21.15.100:0/4170417233 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c01c940 con 0x7f8e1415c620 2024-08-17T02:25:29.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.100+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e140c57c0 con 0x7f8e1415c620 2024-08-17T02:25:29.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.104+0000 7f8e0affd700 1 -- 172.21.15.100:0/4170417233 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8e0c01caa0 con 0x7f8e1415c620 2024-08-17T02:25:29.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.104+0000 7f8e0affd700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8df4060860 0x7f8df4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:29.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.104+0000 7f8e0affd700 1 -- 172.21.15.100:0/4170417233 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f8e0c07e600 con 0x7f8e1415c620 2024-08-17T02:25:29.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.104+0000 7f8e12ffd700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8df4060860 0x7f8df4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:29.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.104+0000 7f8e12ffd700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8df4060860 0x7f8df4062d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e04007fd0 tx=0x7f8e04009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:29.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.112+0000 7f8e0affd700 1 -- 172.21.15.100:0/4170417233 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8e0c04ec50 con 0x7f8e1415c620 2024-08-17T02:25:29.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.272+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e140643a0 con 0x7f8df4060860 2024-08-17T02:25:29.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e0affd700 1 -- 172.21.15.100:0/4170417233 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49244 (secure 0 0 0) 0x7f8e140643a0 con 0x7f8df4060860 2024-08-17T02:25:29.282 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:29.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8df4060860 msgr2=0x7f8df4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:29.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8df4060860 0x7f8df4062d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e04007fd0 tx=0x7f8e04009040 comp rx=0 tx=0).stop 2024-08-17T02:25:29.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1415c620 msgr2=0x7f8e1416a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:29.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1415c620 0x7f8e1416a830 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0093f0 tx=0x7f8e0c016000 comp rx=0 tx=0).stop 2024-08-17T02:25:29.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 shutdown_connections 2024-08-17T02:25:29.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f8df4060860 0x7f8df4062d10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:29.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1415c620 0x7f8e1416a830 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:29.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e140b0320 0x7f8e140c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:29.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 --2- 172.21.15.100:0/4170417233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e140ad1b0 0x7f8e1415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:29.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 >> 172.21.15.100:0/4170417233 conn(0x7f8e140ca900 msgr2=0x7f8e140b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:29.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 shutdown_connections 2024-08-17T02:25:29.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:29.280+0000 7f8e13fff700 1 -- 172.21.15.100:0/4170417233 wait complete. 2024-08-17T02:25:29.285 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:25:29.296 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-08-17T02:25:27.305596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2400688,"kb_used_data":5192,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752574032,"statfs":{"total":773094113280,"available":770635808768,"internally_reserved":0,"allocated":5316608,"data_stored":2893680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005954"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-08-17T02:25:20.157962+0000","last_change":"2024-08-17T02:22:01.241117+0000","last_active":"2024-08-17T02:25:20.157962+0000","last_peered":"2024-08-17T02:25:20.157962+0000","last_clean":"2024-08-17T02:25:20.157962+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:25:20.157962+0000","last_undegraded":"2024-08-17T02:25:20.157962+0000","last_fullsized":"2024-08-17T02:25:20.157962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:21:59.913009+0000","last_clean_scrub_stamp":"2024-08-17T02:21:59.913009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:13:50.401091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298448,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073392,"statfs":{"total":96636764160,"available":96331153408,"internally_reserved":0,"allocated":1114112,"data_stored":805898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91600000000000004,"15min":0.89600000000000002},"min":{"1min":0.68899999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.73699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2769999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.80700000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2989999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.875,"15min":0.875},"min":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.175,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.1579999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.84999999999999998,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2290000000000001,"5min":1.306,"15min":1.306},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.98799999999999999,"15min":0.99399999999999999},"min":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2749999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.254}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.98599999999999999},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.242,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.73999999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3839999999999999,"5min":1.897,"15min":1.897},"last":1.115}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.61499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2390000000000001,"5min":1.288,"15min":1.288},"last":1.3400000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.004,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.24,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.206}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.321,"5min":1.413,"15min":1.413},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.161,"15min":1.167},"min":{"1min":0.92600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.4119999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.115,"15min":1.115},"min":{"1min":0.81799999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4610000000000001,"5min":1.55,"15min":1.55},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.135,"15min":1.137},"min":{"1min":0.82399999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3140000000000001,"5min":1.657,"15min":1.657},"last":1.048}]}]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.621,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.089,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.70699999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.347,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1919999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90500000000000003,"15min":0.90800000000000003},"min":{"1min":0.69499999999999995,"5min":0.621,"15min":0.621},"max":{"1min":1.1850000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89000000000000001,"15min":0.90300000000000002},"min":{"1min":0.70699999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.234,"15min":1.234},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.79800000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.038},"min":{"1min":0.71199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.252,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.0680000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.93300000000000005,"15min":0.94699999999999995},"min":{"1min":0.66300000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1220000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.123,"15min":1.159},"min":{"1min":0.83699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.1100000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.103,"15min":1.137},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.4199999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.042},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.1240000000000001,"15min":1.1479999999999999},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.3779999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.087}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.212,"15min":1.2509999999999999},"min":{"1min":0.89100000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.405,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2270000000000001,"15min":1.25},"min":{"1min":1.044,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.4970000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.133}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297864,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.95899999999999996},"min":{"1min":0.75,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1910000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81499999999999995,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86899999999999999,"15min":0.89800000000000002},"min":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.248,"5min":1.272,"15min":1.272},"last":0.63100000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90800000000000003,"15min":0.89400000000000002},"min":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.98299999999999998},"min":{"1min":0.67300000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.0649999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97499999999999998,"15min":0.98699999999999999},"min":{"1min":0.71999999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.258,"5min":1.429,"15min":1.429},"last":1.048},{"interface":"front","average":{"1min":1.028,"5min":1.083,"15min":1.0960000000000001},"min":{"1min":0.749,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.75,"15min":1.75},"last":0.86899999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95099999999999996,"15min":0.97699999999999998},"min":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.224,"5min":1.506,"15min":1.506},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.04,"15min":1.0740000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.415,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.68300000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.407,"5min":1.855,"15min":1.855},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0740000000000001,"15min":1.085},"min":{"1min":0.76000000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3600000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.88800000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.73499999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4039999999999999,"5min":1.831,"15min":1.831},"last":1.01},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":1.0609999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.667,"5min":2.036,"15min":2.036},"last":1.0960000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302028,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.89800000000000002,"15min":0.91700000000000004},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1279999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1739999999999999,"5min":1.252,"15min":1.252},"last":0.69099999999999995}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0800000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.64700000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1619999999999999,"5min":1.357,"15min":1.357},"last":1.0049999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2410000000000001,"5min":1.516,"15min":1.516},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.72399999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2170000000000001,"5min":1.657,"15min":1.657},"last":0.83399999999999996}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.69299999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2190000000000001,"5min":1.625,"15min":1.625},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.0469999999999999,"15min":1.0449999999999999},"min":{"1min":0.86099999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2989999999999999,"5min":1.444,"15min":1.444},"last":0.97299999999999998}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.99199999999999999,"15min":1.0409999999999999},"min":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0820000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.163,"15min":1.173},"min":{"1min":0.95099999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.095}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.875,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.2969999999999999,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.1220000000000001,"15min":1.1499999999999999},"min":{"1min":0.71899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.286,"5min":1.474,"15min":1.474},"last":0.86699999999999999}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.169,"15min":1.2270000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.26,"5min":3.028,"15min":3.028},"last":1.071},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1839999999999999,"15min":1.202},"min":{"1min":0.998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.28,"5min":1.722,"15min":1.722},"last":1.2}]}]},{"osd":3,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302024,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.083},{"interface":"front","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.70999999999999996},"min":{"1min":0.63900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0720000000000001,"5min":1.105,"15min":1.105},"last":0.96799999999999997}]},{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.622,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":1.056,"15min":1.056},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.65800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":1.327,"15min":1.327},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.94599999999999995,"15min":0.94199999999999995},"min":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.056},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.1899999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.0469999999999999,"15min":1.0569999999999999},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.2,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.105},{"interface":"front","average":{"1min":1.054,"5min":1.0840000000000001,"15min":1.0960000000000001},"min":{"1min":0.92200000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2270000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.407}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0600000000000001},"min":{"1min":0.81999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.27,"5min":1.504,"15min":1.504},"last":1.236},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.1679999999999999},"min":{"1min":0.876,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.3320000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.375}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.92700000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.3109999999999999,"5min":1.784,"15min":1.784},"last":1.552},{"interface":"front","average":{"1min":1.125,"5min":1.103,"15min":1.0880000000000001},"min":{"1min":0.97699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3100000000000001,"5min":1.552,"15min":1.552},"last":1.331}]}]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297864,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0589999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.216,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.252,"5min":1.28,"15min":1.28},"last":0.77900000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.018,"15min":1.02},"min":{"1min":0.78700000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.222,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0449999999999999,"15min":1.0389999999999999},"min":{"1min":0.84099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4319999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.1000000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0609999999999999,"15min":1.0700000000000001},"min":{"1min":0.89000000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.413,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.123},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1080000000000001,"15min":1.115},"min":{"1min":0.86599999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4970000000000001,"5min":1.712,"15min":1.712},"last":1.0209999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1160000000000001,"15min":1.1419999999999999},"min":{"1min":0.86899999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.55,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.155},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.139,"15min":1.143},"min":{"1min":0.79700000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5800000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.331}]},{"osd":7,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.79000000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6180000000000001,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.208,"15min":1.218},"min":{"1min":1,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.575,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.224}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75600000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58099999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92300000000000004,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76700000000000002,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78300000000000003,"15min":0.77100000000000002},"min":{"1min":0.67600000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98899999999999999,"5min":1.421,"15min":1.421},"last":1.1519999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85999999999999999,"15min":0.85199999999999998},"min":{"1min":0.69099999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.90900000000000003,"15min":0.94199999999999995},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2,"5min":3.323,"15min":3.323},"last":0.51500000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.98299999999999998,"15min":1.008},"min":{"1min":0.81799999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.216,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.204},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.72299999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2809999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.518}]},{"osd":5,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0700000000000001,"15min":1.0680000000000001},"min":{"1min":0.80800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.345,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.4730000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0680000000000001,"15min":1.101},"min":{"1min":0.752,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.1479999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0669999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1140000000000001,"15min":1.1220000000000001},"min":{"1min":0.89700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4199999999999999,"5min":1.464,"15min":1.464},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1299999999999999,"15min":1.157},"min":{"1min":0.873,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2410000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.2689999999999999}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.149,"15min":1.159},"min":{"1min":0.97799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.359,"5min":1.605,"15min":1.605},"last":1.321},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2110000000000001},"min":{"1min":1.0429999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.351,"5min":1.635,"15min":1.635},"last":1.615}]}]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298444,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073396,"statfs":{"total":96636764160,"available":96331157504,"internally_reserved":0,"allocated":1114112,"data_stored":805898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.51500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0720000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.747},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.53200000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.288,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.2749999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0549999999999999,"5min":1.129,"15min":1.129},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.83099999999999996,"15min":0.81399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0920000000000001,"5min":1.105,"15min":1.105},"last":0.85499999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93000000000000005,"15min":0.94399999999999995},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.147,"5min":1.165,"15min":1.165},"last":1.3720000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.079,"15min":1.1240000000000001},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.294,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1399999999999999,"15min":1.1879999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.387,"5min":1.573,"15min":1.573},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1879999999999999,"15min":1.214},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.419,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.24}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1160000000000001,"15min":1.117},"min":{"1min":0.877,"5min":0.872,"15min":0.872},"max":{"1min":1.4770000000000001,"5min":1.655,"15min":1.655},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.71499999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.357,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.04}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.109},"min":{"1min":0.83399999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.181},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1899999999999999,"15min":1.224},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.411,"5min":1.633,"15min":1.633},"last":1.1519999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:25:29.301 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-17T02:21:59.913009+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=17, tm_hour=2, tm_min=21, tm_sec=59, tm_wday=5, tm_yday=230, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=17, tm_hour=2, tm_min=25, tm_sec=24, tm_wday=5, tm_yday=230, tm_isdst=0) 2024-08-17T02:25:29.301 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-17T02:25:39.084 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:39.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:49.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:25:49.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb636b700 1 Processor -- start 2024-08-17T02:25:49.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb636b700 1 -- start start 2024-08-17T02:25:49.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb636b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 0x7f6cb00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb636b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6cb015d5b0 0x7f6cb00d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb636b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb00cf190 con 0x7f6cb00d0010 2024-08-17T02:25:49.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb636b700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb00d09a0 con 0x7f6cb015d5b0 2024-08-17T02:25:49.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb636b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00ce5e0 con 0x7f6cb0159040 2024-08-17T02:25:49.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb5b6a700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6cb015d5b0 0x7f6cb00d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb5b6a700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6cb015d5b0 0x7f6cb00d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54914/0 (socket says 172.21.15.100:54914) 2024-08-17T02:25:49.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb5b6a700 1 -- 172.21.15.100:0/3294720737 learned_addr learned my addr 172.21.15.100:0/3294720737 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:49.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.411+0000 7f6cb5369700 1 --2- 172.21.15.100:0/3294720737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 0x7f6cb00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6ca7fff700 1 -- 172.21.15.100:0/3294720737 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811898438 0 0) 0x7f6cb00cf190 con 0x7f6cb00d0010 2024-08-17T02:25:49.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb5b6a700 1 -- 172.21.15.100:0/3294720737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 msgr2=0x7f6cb00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:49.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb5b6a700 1 --2- 172.21.15.100:0/3294720737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 0x7f6cb00cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb5b6a700 1 -- 172.21.15.100:0/3294720737 >> v1:172.21.15.123:6789/0 conn(0x7f6cb00d0010 legacy=0x7f6cb015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb5b6a700 1 -- 172.21.15.100:0/3294720737 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb00ce860 con 0x7f6cb015d5b0 2024-08-17T02:25:49.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb5b6a700 1 --2- 172.21.15.100:0/3294720737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb015d5b0 0x7f6cb00d0460 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009f20 tx=0x7f6ca8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76eb177dacfc06a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:49.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6ca7fff700 1 -- 172.21.15.100:0/3294720737 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800e070 con 0x7f6cb015d5b0 2024-08-17T02:25:49.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6ca7fff700 1 -- 172.21.15.100:0/3294720737 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca800b9d0 con 0x7f6cb015d5b0 2024-08-17T02:25:49.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6ca7fff700 1 -- 172.21.15.100:0/3294720737 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80092e0 con 0x7f6cb015d5b0 2024-08-17T02:25:49.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 -- 172.21.15.100:0/3294720737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb015d5b0 msgr2=0x7f6cb00d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 --2- 172.21.15.100:0/3294720737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb015d5b0 0x7f6cb00d0460 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009f20 tx=0x7f6ca8009840 comp rx=0 tx=0).stop 2024-08-17T02:25:49.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 -- 172.21.15.100:0/3294720737 shutdown_connections 2024-08-17T02:25:49.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 --2- 172.21.15.100:0/3294720737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb015d5b0 0x7f6cb00d0460 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 --2- 172.21.15.100:0/3294720737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 0x7f6cb00cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 -- 172.21.15.100:0/3294720737 >> 172.21.15.100:0/3294720737 conn(0x7f6cb0068250 msgr2=0x7f6cb0068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:49.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 -- 172.21.15.100:0/3294720737 shutdown_connections 2024-08-17T02:25:49.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 -- 172.21.15.100:0/3294720737 wait complete. 2024-08-17T02:25:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 Processor -- start 2024-08-17T02:25:49.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.415+0000 7f6cb636b700 1 -- start start 2024-08-17T02:25:49.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb00d0010 0x7f6cb01693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 0x7f6cb015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb015d5b0 0x7f6cb0152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00afef0 con 0x7f6cb00d0010 2024-08-17T02:25:49.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00b01d0 con 0x7f6cb015d5b0 2024-08-17T02:25:49.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb015b0d0 con 0x7f6cb0159040 2024-08-17T02:25:49.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb00d0010 0x7f6cb01693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb00d0010 0x7f6cb01693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54926/0 (socket says 172.21.15.100:54926) 2024-08-17T02:25:49.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 -- 172.21.15.100:0/1528758804 learned_addr learned my addr 172.21.15.100:0/1528758804 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:49.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5b6a700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb015d5b0 0x7f6cb0152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb4b68700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 0x7f6cb015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 -- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 msgr2=0x7f6cb015a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 0x7f6cb015a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 -- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb015d5b0 msgr2=0x7f6cb0152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb015d5b0 0x7f6cb0152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 -- 172.21.15.100:0/1528758804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8009ae0 con 0x7f6cb00d0010 2024-08-17T02:25:49.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb5369700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb00d0010 0x7f6cb01693d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000ecb0 tx=0x7f6ca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:49.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6ca57fa700 1 -- 172.21.15.100:0/1528758804 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000c850 con 0x7f6cb00d0010 2024-08-17T02:25:49.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6ca57fa700 1 -- 172.21.15.100:0/1528758804 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca000c9b0 con 0x7f6cb00d0010 2024-08-17T02:25:49.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6ca57fa700 1 -- 172.21.15.100:0/1528758804 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000cd50 con 0x7f6cb00d0010 2024-08-17T02:25:49.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb015b3b0 con 0x7f6cb00d0010 2024-08-17T02:25:49.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb0169910 con 0x7f6cb00d0010 2024-08-17T02:25:49.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.419+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb015b830 con 0x7f6cb00d0010 2024-08-17T02:25:49.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.423+0000 7f6ca57fa700 1 -- 172.21.15.100:0/1528758804 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6ca0013680 con 0x7f6cb00d0010 2024-08-17T02:25:49.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.423+0000 7f6ca57fa700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c94060860 0x7f6c94062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.423+0000 7f6ca57fa700 1 -- 172.21.15.100:0/1528758804 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f6ca007cb60 con 0x7f6cb00d0010 2024-08-17T02:25:49.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.423+0000 7f6cb4b68700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c94060860 0x7f6c94062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.423+0000 7f6cb4b68700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c94060860 0x7f6c94062d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6cb015bf00 tx=0x7f6cac007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:49.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.431+0000 7f6ca57fa700 1 -- 172.21.15.100:0/1528758804 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6ca004d1b0 con 0x7f6cb00d0010 2024-08-17T02:25:49.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.591+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 --> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cb00b0310 con 0x7f6c94060860 2024-08-17T02:25:49.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.595+0000 7f6ca57fa700 1 -- 172.21.15.100:0/1528758804 <== mgr.4102 v2:172.21.15.123:6832/24094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49142 (secure 0 0 0) 0x7f6cac002650 con 0x7f6c94060860 2024-08-17T02:25:49.600 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:49.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c94060860 msgr2=0x7f6c94062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c94060860 0x7f6c94062d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6cb015bf00 tx=0x7f6cac007ec0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb00d0010 msgr2=0x7f6cb01693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb00d0010 0x7f6cb01693d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000ecb0 tx=0x7f6ca00091a0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 shutdown_connections 2024-08-17T02:25:49.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f6c94060860 0x7f6c94062d10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb015d5b0 0x7f6cb0152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cb0159040 0x7f6cb015a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 --2- 172.21.15.100:0/1528758804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cb00d0010 0x7f6cb01693d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.599+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 >> 172.21.15.100:0/1528758804 conn(0x7f6cb0068250 msgr2=0x7f6cb0159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:49.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.603+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 shutdown_connections 2024-08-17T02:25:49.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.603+0000 7f6cb636b700 1 -- 172.21.15.100:0/1528758804 wait complete. 2024-08-17T02:25:49.606 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-08-17T02:25:49.619 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-08-17T02:25:49.311785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2400708,"kb_used_data":5224,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752574012,"statfs":{"total":773094113280,"available":770635788288,"internally_reserved":0,"allocated":5349376,"data_stored":2904976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003349"},"pg_stats":[{"pgid":"1.0","version":"44'119","reported_seq":196,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-08-17T02:25:34.335402+0000","last_change":"2024-08-17T02:25:34.335341+0000","last_active":"2024-08-17T02:25:34.335402+0000","last_peered":"2024-08-17T02:25:34.335402+0000","last_clean":"2024-08-17T02:25:34.335402+0000","last_became_active":"2024-08-17T02:22:01.238561+0000","last_became_peered":"2024-08-17T02:22:01.238561+0000","last_unstale":"2024-08-17T02:25:34.335402+0000","last_undegraded":"2024-08-17T02:25:34.335402+0000","last_fullsized":"2024-08-17T02:25:34.335402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'119","last_scrub_stamp":"2024-08-17T02:25:34.335224+0000","last_deep_scrub":"44'119","last_deep_scrub_stamp":"2024-08-17T02:25:34.335224+0000","last_clean_scrub_stamp":"2024-08-17T02:25:34.335224+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:12:12.077438+0000","scrub_duration":0.013445576000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298440,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073400,"statfs":{"total":96636764160,"available":96331161600,"internally_reserved":0,"allocated":1114112,"data_stored":805898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91600000000000004,"15min":0.89600000000000002},"min":{"1min":0.68899999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":1.129},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.73699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2769999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.071}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2989999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.875,"15min":0.875},"min":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.175,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.94599999999999995}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.84999999999999998,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2290000000000001,"5min":1.306,"15min":1.306},"last":1.093},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.98799999999999999,"15min":0.99399999999999999},"min":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2749999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.0429999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.98599999999999999},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.242,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.73999999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3839999999999999,"5min":1.897,"15min":1.897},"last":1.2909999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.61499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2390000000000001,"5min":1.288,"15min":1.288},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.004,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.24,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.1779999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.321,"5min":1.413,"15min":1.413},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.161,"15min":1.167},"min":{"1min":0.92600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.2030000000000001}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.115,"15min":1.115},"min":{"1min":0.81799999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4610000000000001,"5min":1.55,"15min":1.55},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.135,"15min":1.137},"min":{"1min":0.82399999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3140000000000001,"5min":1.657,"15min":1.657},"last":1.2609999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302024,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.621,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.089,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.70699999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.347,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.0649999999999999}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1919999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.044},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90500000000000003,"15min":0.90800000000000003},"min":{"1min":0.69499999999999995,"5min":0.621,"15min":0.621},"max":{"1min":1.1850000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.745}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89000000000000001,"15min":0.90300000000000002},"min":{"1min":0.70699999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.234,"15min":1.234},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.99299999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.79800000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.038},"min":{"1min":0.71199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.252,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.95799999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.93300000000000005,"15min":0.94699999999999995},"min":{"1min":0.66300000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1220000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.107},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.123,"15min":1.159},"min":{"1min":0.83699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.103,"15min":1.137},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.4199999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.1240000000000001,"15min":1.1479999999999999},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.3779999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.89000000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.212,"15min":1.2509999999999999},"min":{"1min":0.89100000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.405,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.149},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2270000000000001,"15min":1.25},"min":{"1min":1.044,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.4970000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.089}]}]},{"osd":5,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297864,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.95899999999999996},"min":{"1min":0.75,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1910000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.96699999999999997}]},{"osd":1,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81499999999999995,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86899999999999999,"15min":0.89800000000000002},"min":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.248,"5min":1.272,"15min":1.272},"last":0.94199999999999995}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90800000000000003,"15min":0.89400000000000002},"min":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.98299999999999998},"min":{"1min":0.67300000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.0029999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97499999999999998,"15min":0.98699999999999999},"min":{"1min":0.71999999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.258,"5min":1.429,"15min":1.429},"last":1.149},{"interface":"front","average":{"1min":1.028,"5min":1.083,"15min":1.0960000000000001},"min":{"1min":0.749,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.75,"15min":1.75},"last":1.079}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95099999999999996,"15min":0.97699999999999998},"min":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.224,"5min":1.506,"15min":1.506},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.04,"15min":1.0740000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.415,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.032}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.68300000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.407,"5min":1.855,"15min":1.855},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0740000000000001,"15min":1.085},"min":{"1min":0.76000000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3600000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.73499999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4039999999999999,"5min":1.831,"15min":1.831},"last":1.216},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":1.0609999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.667,"5min":2.036,"15min":2.036},"last":1.131}]}]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302028,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.89800000000000002,"15min":0.91700000000000004},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1279999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1739999999999999,"5min":1.252,"15min":1.252},"last":1.1459999999999999}]},{"osd":1,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0800000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.64700000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1619999999999999,"5min":1.357,"15min":1.357},"last":1.0600000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2410000000000001,"5min":1.516,"15min":1.516},"last":0.872},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.72399999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2170000000000001,"5min":1.657,"15min":1.657},"last":0.72499999999999998}]},{"osd":3,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.69299999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2190000000000001,"5min":1.625,"15min":1.625},"last":1.103},{"interface":"front","average":{"1min":1.079,"5min":1.0469999999999999,"15min":1.0449999999999999},"min":{"1min":0.86099999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2989999999999999,"5min":1.444,"15min":1.444},"last":0.91700000000000004}]},{"osd":5,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.99199999999999999,"15min":1.0409999999999999},"min":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0820000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.101,"5min":1.163,"15min":1.173},"min":{"1min":0.95099999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.169}]},{"osd":6,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.875,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.2969999999999999,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.1220000000000001,"15min":1.1499999999999999},"min":{"1min":0.71899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.286,"5min":1.474,"15min":1.474},"last":1.024}]},{"osd":7,"last update":"Sat Aug 17 02:25:02 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.169,"15min":1.2270000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.26,"5min":3.028,"15min":3.028},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1839999999999999,"15min":1.202},"min":{"1min":0.998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.28,"5min":1.722,"15min":1.722},"last":1.2090000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302024,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.70999999999999996},"min":{"1min":0.63900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0720000000000001,"5min":1.105,"15min":1.105},"last":0.76100000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.622,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":1.056,"15min":1.056},"last":0.72299999999999998}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.65800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":1.327,"15min":1.327},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.84599999999999997}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.94599999999999995,"15min":0.94199999999999995},"min":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.056},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.085}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.0469999999999999,"15min":1.0569999999999999},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.2,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.0840000000000001,"15min":1.0960000000000001},"min":{"1min":0.92200000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2270000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1339999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0600000000000001},"min":{"1min":0.81999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.27,"5min":1.504,"15min":1.504},"last":1.272},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.1679999999999999},"min":{"1min":0.876,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.3320000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.3580000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.92700000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.3109999999999999,"5min":1.784,"15min":1.784},"last":1.056},{"interface":"front","average":{"1min":1.125,"5min":1.103,"15min":1.0880000000000001},"min":{"1min":0.97699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3100000000000001,"5min":1.552,"15min":1.552},"last":1.3320000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297864,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76300000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0589999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.216,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.252,"5min":1.28,"15min":1.28},"last":1.018}]},{"osd":4,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.018,"15min":1.02},"min":{"1min":0.78700000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.222,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.129},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0449999999999999,"15min":1.0389999999999999},"min":{"1min":0.84099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4319999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.995}]},{"osd":5,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0609999999999999,"15min":1.0700000000000001},"min":{"1min":0.89000000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.413,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1080000000000001,"15min":1.115},"min":{"1min":0.86599999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4970000000000001,"5min":1.712,"15min":1.712},"last":0.876}]},{"osd":6,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1160000000000001,"15min":1.1419999999999999},"min":{"1min":0.86899999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.55,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.139,"15min":1.143},"min":{"1min":0.79700000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5800000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.0840000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:25:08 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.79000000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6180000000000001,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.159},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.208,"15min":1.218},"min":{"1min":1,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.575,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2}]}]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302024,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":520192,"data_stored":215530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75600000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58099999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92300000000000004,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.60499999999999998}]},{"osd":2,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76700000000000002,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.502},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78300000000000003,"15min":0.77100000000000002},"min":{"1min":0.67600000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98899999999999999,"5min":1.421,"15min":1.421},"last":1.0149999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85999999999999999,"15min":0.85199999999999998},"min":{"1min":0.69099999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.90900000000000003,"15min":0.94199999999999995},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2,"5min":3.323,"15min":3.323},"last":0.873}]},{"osd":4,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.98299999999999998,"15min":1.008},"min":{"1min":0.81799999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.216,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.72299999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2809999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.2170000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0700000000000001,"15min":1.0680000000000001},"min":{"1min":0.80800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.345,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0680000000000001,"15min":1.101},"min":{"1min":0.752,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.1479999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.91500000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1140000000000001,"15min":1.1220000000000001},"min":{"1min":0.89700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4199999999999999,"5min":1.464,"15min":1.464},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1299999999999999,"15min":1.157},"min":{"1min":0.873,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2410000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.97499999999999998}]},{"osd":7,"last update":"Sat Aug 17 02:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.149,"15min":1.159},"min":{"1min":0.97799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.359,"5min":1.605,"15min":1.605},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2110000000000001},"min":{"1min":1.0429999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.351,"5min":1.635,"15min":1.635},"last":1.1890000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298440,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073400,"statfs":{"total":96636764160,"available":96331161600,"internally_reserved":0,"allocated":1114112,"data_stored":805898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.51500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85199999999999998}]},{"osd":2,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0720000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.53200000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.288,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.77800000000000002}]},{"osd":3,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0549999999999999,"5min":1.129,"15min":1.129},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.83099999999999996,"15min":0.81399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0920000000000001,"5min":1.105,"15min":1.105},"last":0.70899999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93000000000000005,"15min":0.94399999999999995},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.147,"5min":1.165,"15min":1.165},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.079,"15min":1.1240000000000001},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.294,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.91700000000000004}]},{"osd":5,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1399999999999999,"15min":1.1879999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.387,"5min":1.573,"15min":1.573},"last":0.998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1879999999999999,"15min":1.214},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.419,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.028}]},{"osd":6,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1160000000000001,"15min":1.117},"min":{"1min":0.877,"5min":0.872,"15min":0.872},"max":{"1min":1.4770000000000001,"5min":1.655,"15min":1.655},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.71499999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.357,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.054}]},{"osd":7,"last update":"Sat Aug 17 02:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.109},"min":{"1min":0.83399999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.101},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1899999999999999,"15min":1.224},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.411,"5min":1.633,"15min":1.633},"last":1.1180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:25:49.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-08-17T02:25:49.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1ef155c700 1 Processor -- start 2024-08-17T02:25:49.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1ef155c700 1 -- start start 2024-08-17T02:25:49.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1ef155c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1eec159480 0x7f1eec0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1ef155c700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec0ce210 con 0x7f1eec1590b0 2024-08-17T02:25:49.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1ef155c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec0cb440 con 0x7f1eec067510 2024-08-17T02:25:49.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1ef155c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec0cba00 con 0x7f1eec159480 2024-08-17T02:25:49.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1eebfff700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1eec1590b0 0x7f1eec066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:54106/0 (socket says 172.21.15.100:54106) 2024-08-17T02:25:49.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1eebfff700 1 -- 172.21.15.100:0/1002322026 learned_addr learned my addr 172.21.15.100:0/1002322026 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:49.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.723+0000 7f1ef0d5b700 1 --2- 172.21.15.100:0/1002322026 >> v2:172.21.15.123:3300/0 conn(0x7f1eec159480 0x7f1eec0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969934266 0 0) 0x7f1eec0ce210 con 0x7f1eec1590b0 2024-08-17T02:25:49.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed4003570 con 0x7f1eec1590b0 2024-08-17T02:25:49.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257406097 0 0) 0x7f1eec0cb440 con 0x7f1eec067510 2024-08-17T02:25:49.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eec0ce210 con 0x7f1eec067510 2024-08-17T02:25:49.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328069774 0 0) 0x7f1ed4003570 con 0x7f1eec1590b0 2024-08-17T02:25:49.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 >> v2:172.21.15.123:3300/0 conn(0x7f1eec159480 msgr2=0x7f1eec0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:49.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 --2- 172.21.15.100:0/1002322026 >> v2:172.21.15.123:3300/0 conn(0x7f1eec159480 0x7f1eec0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 >> v1:172.21.15.123:6789/0 conn(0x7f1eec067510 legacy=0x7f1eec0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0cb720 con 0x7f1eec1590b0 2024-08-17T02:25:49.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3387233635 0 0) 0x7f1ee0001470 con 0x7f1eec1590b0 2024-08-17T02:25:49.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 >> v1:172.21.15.100:6789/0 conn(0x7f1eec1590b0 legacy=0x7f1eec066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4003bf0 0x7f1ed4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ed4004810 0x7f1ed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ed40051c0 0x7f1ed400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed400c080 con 0x7f1ed4004810 2024-08-17T02:25:49.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed400c1f0 con 0x7f1ed4003bf0 2024-08-17T02:25:49.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed400c3b0 con 0x7f1ed40051c0 2024-08-17T02:25:49.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeb7fe700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ed4004810 0x7f1ed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eebfff700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4003bf0 0x7f1ed4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef0d5b700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ed40051c0 0x7f1ed400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeb7fe700 1 -- 172.21.15.100:0/1002322026 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ed40051c0 msgr2=0x7f1ed400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:49.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeb7fe700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ed40051c0 0x7f1ed400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeb7fe700 1 -- 172.21.15.100:0/1002322026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4003bf0 msgr2=0x7f1ed4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:25:49.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeb7fe700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4003bf0 0x7f1ed4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef0d5b700 1 -- 172.21.15.100:0/1002322026 reap_dead start 2024-08-17T02:25:49.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeb7fe700 1 -- 172.21.15.100:0/1002322026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed4003570 con 0x7f1ed4004810 2024-08-17T02:25:49.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeb7fe700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ed4004810 0x7f1ed4004c80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1edc001a10 tx=0x7f1edc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15a6f914b3f1bcb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:49.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00c590 con 0x7f1ed4004810 2024-08-17T02:25:49.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00c6f0 con 0x7f1ed4004810 2024-08-17T02:25:49.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1eeaffd700 1 -- 172.21.15.100:0/1002322026 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00ca90 con 0x7f1ed4004810 2024-08-17T02:25:49.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef155c700 1 -- 172.21.15.100:0/1002322026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ed4004810 msgr2=0x7f1ed4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef155c700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ed4004810 0x7f1ed4004c80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1edc001a10 tx=0x7f1edc007d30 comp rx=0 tx=0).stop 2024-08-17T02:25:49.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef155c700 1 -- 172.21.15.100:0/1002322026 shutdown_connections 2024-08-17T02:25:49.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef155c700 1 --2- 172.21.15.100:0/1002322026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ed4004810 0x7f1ed4004c80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef155c700 1 -- 172.21.15.100:0/1002322026 >> 172.21.15.100:0/1002322026 conn(0x7f1eec0d0b90 msgr2=0x7f1eec0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:49.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef155c700 1 -- 172.21.15.100:0/1002322026 shutdown_connections 2024-08-17T02:25:49.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.727+0000 7f1ef155c700 1 -- 172.21.15.100:0/1002322026 wait complete. 2024-08-17T02:25:49.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 Processor -- start 2024-08-17T02:25:49.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 -- start start 2024-08-17T02:25:49.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4004810 0x7f1eec143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eec169bf0 0x7f1eec153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1eec153780 0x7f1eec0c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec143ea0 con 0x7f1eec169bf0 2024-08-17T02:25:49.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec152bd0 con 0x7f1ed4004810 2024-08-17T02:25:49.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec153bb0 con 0x7f1eec153780 2024-08-17T02:25:49.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eec169bf0 0x7f1eec153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eec169bf0 0x7f1eec153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54934/0 (socket says 172.21.15.100:54934) 2024-08-17T02:25:49.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 -- 172.21.15.100:0/1906316569 learned_addr learned my addr 172.21.15.100:0/1906316569 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:49.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef0d5b700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1eec153780 0x7f1eec0c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eebfff700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4004810 0x7f1eec143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 -- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1eec153780 msgr2=0x7f1eec0c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1eec153780 0x7f1eec0c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 -- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4004810 msgr2=0x7f1eec143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4004810 0x7f1eec143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 -- 172.21.15.100:0/1906316569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec151fc0 con 0x7f1eec169bf0 2024-08-17T02:25:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1eeb7fe700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eec169bf0 0x7f1eec153240 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c3c0 tx=0x7f1edc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ed3fff700 1 -- 172.21.15.100:0/1906316569 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00c590 con 0x7f1eec169bf0 2024-08-17T02:25:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ed3fff700 1 -- 172.21.15.100:0/1906316569 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00ce40 con 0x7f1eec169bf0 2024-08-17T02:25:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec1522a0 con 0x7f1eec169bf0 2024-08-17T02:25:49.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ed3fff700 1 -- 172.21.15.100:0/1906316569 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc007b10 con 0x7f1eec169bf0 2024-08-17T02:25:49.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.731+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec152860 con 0x7f1eec169bf0 2024-08-17T02:25:49.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.735+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec152d60 con 0x7f1eec169bf0 2024-08-17T02:25:49.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.735+0000 7f1ed3fff700 1 -- 172.21.15.100:0/1906316569 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1edc019030 con 0x7f1eec169bf0 2024-08-17T02:25:49.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.735+0000 7f1ed3fff700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1ed8060890 0x7f1ed8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:49.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.735+0000 7f1ed3fff700 1 -- 172.21.15.100:0/1906316569 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f1edc07de90 con 0x7f1eec169bf0 2024-08-17T02:25:49.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.735+0000 7f1eebfff700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1ed8060890 0x7f1ed8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:49.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.739+0000 7f1eebfff700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1ed8060890 0x7f1ed8062d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0001660 tx=0x7f1ee0001df0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.747+0000 7f1ed3fff700 1 -- 172.21.15.100:0/1906316569 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1edc04e580 con 0x7f1eec169bf0 2024-08-17T02:25:49.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.911+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f1eec064330 con 0x7f1eec169bf0 2024-08-17T02:25:49.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.915+0000 7f1ed3fff700 1 -- 172.21.15.100:0/1906316569 <== mon.0 v2:172.21.15.100:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1edc04e580 con 0x7f1eec169bf0 2024-08-17T02:25:49.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ed3fff700 1 -- 172.21.15.100:0/1906316569 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f1edc0220c0 con 0x7f1eec169bf0 2024-08-17T02:25:49.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1ed8060890 msgr2=0x7f1ed8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1ed8060890 0x7f1ed8062d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0001660 tx=0x7f1ee0001df0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eec169bf0 msgr2=0x7f1eec153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:49.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eec169bf0 0x7f1eec153240 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c3c0 tx=0x7f1edc015000 comp rx=0 tx=0).stop 2024-08-17T02:25:49.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 shutdown_connections 2024-08-17T02:25:49.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7f1ed8060890 0x7f1ed8062d40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1eec153780 0x7f1eec0c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eec169bf0 0x7f1eec153240 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 --2- 172.21.15.100:0/1906316569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed4004810 0x7f1eec143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:49.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 >> 172.21.15.100:0/1906316569 conn(0x7f1eec0d0b90 msgr2=0x7f1eec15a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:49.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 shutdown_connections 2024-08-17T02:25:49.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:49.919+0000 7f1ef155c700 1 -- 172.21.15.100:0/1906316569 wait complete. 2024-08-17T02:25:49.935 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-17T02:25:49.936 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:25:50.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 Processor -- start 2024-08-17T02:25:50.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- start start 2024-08-17T02:25:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd4fc0af5f0 0x7fd4fc0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc0ad8a0 con 0x7fd4fc0ad090 2024-08-17T02:25:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc0adb80 con 0x7fd4fc155d80 2024-08-17T02:25:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc0ade60 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4f9d9b700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd4fc0af5f0 0x7fd4fc0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fad9d700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd4fc0ad090 0x7fd4fc157a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:54122/0 (socket says 172.21.15.100:54122) 2024-08-17T02:25:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fad9d700 1 -- 172.21.15.100:0/317501674 learned_addr learned my addr 172.21.15.100:0/317501674 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-08-17T02:25:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4f9d9b700 1 -- 172.21.15.100:0/317501674 >> v1:172.21.15.123:6789/0 conn(0x7fd4fc155d80 legacy=0x7fd4fc0aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4f9d9b700 1 -- 172.21.15.100:0/317501674 >> v1:172.21.15.100:6789/0 conn(0x7fd4fc0ad090 legacy=0x7fd4fc157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4f9d9b700 1 -- 172.21.15.100:0/317501674 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0ae140 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4f9d9b700 1 --2- 172.21.15.100:0/317501674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc0acb50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009e60 tx=0x7fd4f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc92215b0952a80b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4f959a700 1 -- 172.21.15.100:0/317501674 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400e070 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4f959a700 1 -- 172.21.15.100:0/317501674 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd4f400bb30 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4f959a700 1 -- 172.21.15.100:0/317501674 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4012430 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- 172.21.15.100:0/317501674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 msgr2=0x7fd4fc0acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 --2- 172.21.15.100:0/317501674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc0acb50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009e60 tx=0x7fd4f4009780 comp rx=0 tx=0).stop 2024-08-17T02:25:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- 172.21.15.100:0/317501674 shutdown_connections 2024-08-17T02:25:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 --2- 172.21.15.100:0/317501674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc0acb50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- 172.21.15.100:0/317501674 >> 172.21.15.100:0/317501674 conn(0x7fd4fc06e070 msgr2=0x7fd4fc0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- 172.21.15.100:0/317501674 shutdown_connections 2024-08-17T02:25:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.039+0000 7fd4fb59e700 1 -- 172.21.15.100:0/317501674 wait complete. 2024-08-17T02:25:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fb59e700 1 Processor -- start 2024-08-17T02:25:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fb59e700 1 -- start start 2024-08-17T02:25:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4fc0ad090 0x7fd4fc14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fb59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc16b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4fc155d80 0x7fd4fc16b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fb59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0b0dd0 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4f9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc16b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0b10b0 con 0x7fd4fc155d80 2024-08-17T02:25:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0b1390 con 0x7fd4fc0ad090 2024-08-17T02:25:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4f9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc16b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54944/0 (socket says 172.21.15.100:54944) 2024-08-17T02:25:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4f9d9b700 1 -- 172.21.15.100:0/218523421 learned_addr learned my addr 172.21.15.100:0/218523421 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-08-17T02:25:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fa59c700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4fc0ad090 0x7fd4fc14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.043+0000 7fd4fad9d700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4fc155d80 0x7fd4fc16b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4f9d9b700 1 -- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4fc0ad090 msgr2=0x7fd4fc14ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4f9d9b700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4fc0ad090 0x7fd4fc14ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:50.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4f9d9b700 1 -- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4fc155d80 msgr2=0x7fd4fc16b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:50.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4f9d9b700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4fc155d80 0x7fd4fc16b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:50.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4f9d9b700 1 -- 172.21.15.100:0/218523421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f4009a20 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4f9d9b700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc16b1e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc06fbe0 tx=0x7fd4f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:50.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/218523421 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4012e00 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/218523421 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd4f4025460 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc0b1670 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/218523421 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4025840 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc148e40 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.047+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc0638f0 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.051+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/218523421 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd4f4025bf0 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.051+0000 7fd4ea7fc700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd4dc060860 0x7fd4dc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:25:50.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.051+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/218523421 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fd4f407e920 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.051+0000 7fd4fa59c700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd4dc060860 0x7fd4dc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:25:50.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.051+0000 7fd4fa59c700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd4dc060860 0x7fd4dc062d10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0003a80 tx=0x7fd4f0007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:25:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.055+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/218523421 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd4f404f090 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.255+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4fc0aeaf0 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.255+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/218523421 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fd4f4020df0 con 0x7fd4fc0af5f0 2024-08-17T02:25:50.259 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-08-17T02:25:50.260 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4075860098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4075860098},{"type":"v1","addr":"172.21.15.123:6835","nonce":4075860098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/4238364994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":4238364994},{"type":"v1","addr":"172.21.15.123:6837","nonce":4238364994}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1877360383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1877360383},{"type":"v1","addr":"172.21.15.100:6835","nonce":1877360383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4075211675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4075211675},{"type":"v1","addr":"172.21.15.100:6833","nonce":4075211675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[]} 2024-08-17T02:25:50.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd4dc060860 msgr2=0x7fd4dc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:50.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd4dc060860 0x7fd4dc062d10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0003a80 tx=0x7fd4f0007ec0 comp rx=0 tx=0).stop 2024-08-17T02:25:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 msgr2=0x7fd4fc16b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:25:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc16b1e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc06fbe0 tx=0x7fd4f4009780 comp rx=0 tx=0).stop 2024-08-17T02:25:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 shutdown_connections 2024-08-17T02:25:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:6832/24094,v1:172.21.15.123:6833/24094] conn(0x7fd4dc060860 0x7fd4dc062d10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4fc155d80 0x7fd4fc16b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc0af5f0 0x7fd4fc16b1e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 --2- 172.21.15.100:0/218523421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4fc0ad090 0x7fd4fc14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:25:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 >> 172.21.15.100:0/218523421 conn(0x7fd4fc06e070 msgr2=0x7fd4fc06e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:25:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 shutdown_connections 2024-08-17T02:25:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-08-17T02:25:50.259+0000 7fd4fb59e700 1 -- 172.21.15.100:0/218523421 wait complete. 2024-08-17T02:25:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 20 2024-08-17T02:25:50.274 DEBUG:tasks.cephfs.filesystem:already dead... 2024-08-17T02:25:50.274 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-17T02:25:50.274 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-17T02:25:50.274 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.287 INFO:tasks.ceph.mds.a:Stopped 2024-08-17T02:25:50.288 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-17T02:25:50.288 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.300 INFO:tasks.ceph.mds.c:Stopped 2024-08-17T02:25:50.301 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-17T02:25:50.301 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.317 INFO:tasks.ceph.mds.b:Stopped 2024-08-17T02:25:50.317 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-17T02:25:50.317 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.327 INFO:tasks.ceph.mds.d:Stopped 2024-08-17T02:25:50.328 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-17T02:25:50.328 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-08-17T02:25:50.328 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.384 INFO:tasks.ceph.osd.0:Stopped 2024-08-17T02:25:50.384 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-08-17T02:25:50.384 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.454 INFO:tasks.ceph.osd.1:Stopped 2024-08-17T02:25:50.454 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-08-17T02:25:50.454 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.541 INFO:tasks.ceph.osd.2:Stopped 2024-08-17T02:25:50.542 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-08-17T02:25:50.542 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.597 INFO:tasks.ceph.osd.3:Stopped 2024-08-17T02:25:50.598 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-08-17T02:25:50.598 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.673 INFO:tasks.ceph.osd.4:Stopped 2024-08-17T02:25:50.673 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-08-17T02:25:50.673 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.745 INFO:tasks.ceph.osd.5:Stopped 2024-08-17T02:25:50.746 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-08-17T02:25:50.746 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.826 INFO:tasks.ceph.osd.6:Stopped 2024-08-17T02:25:50.826 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-08-17T02:25:50.826 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.918 INFO:tasks.ceph.osd.7:Stopped 2024-08-17T02:25:50.918 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-17T02:25:50.919 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-08-17T02:25:50.919 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.949 INFO:tasks.ceph.mgr.y:Stopped 2024-08-17T02:25:50.949 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-08-17T02:25:50.949 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.974 INFO:tasks.ceph.mgr.x:Stopped 2024-08-17T02:25:50.974 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-17T02:25:50.974 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-08-17T02:25:50.975 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:50.994 INFO:tasks.ceph.mon.a:Stopped 2024-08-17T02:25:50.995 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-17T02:25:50.995 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:51.011 INFO:tasks.ceph.mon.b:Stopped 2024-08-17T02:25:51.011 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-17T02:25:51.011 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:25:51.072 INFO:tasks.ceph.mon.c:Stopped 2024-08-17T02:25:51.072 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-17T02:25:51.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-08-17T02:25:51.096 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:25:51.096 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-08-17T02:25:51.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:25:51.384 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-08-17T02:25:51.580 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:25:51.581 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-08-17T02:25:51.764 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:25:51.764 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-08-17T02:25:51.958 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:25:51.958 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-08-17T02:25:52.270 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:25:52.270 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-08-17T02:25:52.527 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:25:52.528 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-08-17T02:25:52.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:25:52.711 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-08-17T02:25:52.935 INFO:tasks.ceph:Archiving mon data... 2024-08-17T02:25:52.936 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/data/mon.a.tgz 2024-08-17T02:25:52.936 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-08-17T02:25:52.943 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.cFg0VXFzs0 2024-08-17T02:25:52.943 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cFg0VXFzs0 2024-08-17T02:25:53.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.cFg0VXFzs0 2024-08-17T02:25:53.087 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.cFg0VXFzs0 is 307KB 2024-08-17T02:25:53.154 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.cFg0VXFzs0 2024-08-17T02:25:53.161 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/data/mon.b.tgz 2024-08-17T02:25:53.161 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-08-17T02:25:53.167 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.S1kL0pfMop 2024-08-17T02:25:53.168 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.S1kL0pfMop 2024-08-17T02:25:53.314 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.S1kL0pfMop 2024-08-17T02:25:53.336 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.S1kL0pfMop is 408KB 2024-08-17T02:25:53.397 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.S1kL0pfMop 2024-08-17T02:25:53.403 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/data/mon.c.tgz 2024-08-17T02:25:53.404 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-08-17T02:25:53.410 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.k8b8CodFZM 2024-08-17T02:25:53.411 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.k8b8CodFZM 2024-08-17T02:25:53.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.k8b8CodFZM 2024-08-17T02:25:53.578 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.k8b8CodFZM is 408KB 2024-08-17T02:25:53.598 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.k8b8CodFZM 2024-08-17T02:25:53.604 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-17T02:25:53.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-17T02:25:53.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-17T02:25:53.660 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-08-17T02:25:53.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-17T02:25:53.665 DEBUG:teuthology.orchestra.run.smithi123:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-17T02:25:53.697 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-17T02:25:53.698 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-17T02:25:53.705 INFO:tasks.ceph:Archiving crash dumps... 2024-08-17T02:25:53.707 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/remote/smithi100/crash 2024-08-17T02:25:53.708 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-17T02:25:53.725 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/remote/smithi123/crash 2024-08-17T02:25:53.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-17T02:25:53.743 INFO:tasks.ceph:Shutting down logrotate 2024-08-17T02:25:53.743 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:53.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:53.830 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:53.846 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-17T02:25:53.860 INFO:tasks.ceph:Compressing logs... 2024-08-17T02:25:53.861 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-17T02:25:53.863 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-17T02:25:53.882 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-08-17T02:25:53.882 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24346.log 2024-08-17T02:25:53.883 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 /var/log/ceph/ceph-mds.d.log: --verbose -- /var/log/ceph/ceph-mon.b.log 2024-08-17T02:25:53.884 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.24346.log: gzip -5 87.6%--verbose -- replaced with /var/log/ceph/ceph-client.admin.24346.log.gz 2024-08-17T02:25:53.884 INFO:teuthology.orchestra.run.smithi123.stderr:-- /var/log/ceph/ceph-client.1.29612.log 2024-08-17T02:25:53.884 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24211.log 2024-08-17T02:25:53.885 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.1.29612.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-08-17T02:25:53.885 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-08-17T02:25:53.886 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.24211.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-08-17T02:25:53.886 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-mgr.x.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.24211.log.gz 2024-08-17T02:25:53.886 INFO:teuthology.orchestra.run.smithi123.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.1.29612.log.gz 2024-08-17T02:25:53.890 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32203.log 2024-08-17T02:25:53.890 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29424.log 2024-08-17T02:25:53.891 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31356.log 2024-08-17T02:25:53.891 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32203.log: /var/log/ceph/ceph-client.admin.29424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29854.log 2024-08-17T02:25:53.895 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24320.log 2024-08-17T02:25:53.895 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-17T02:25:53.895 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.24320.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.24320.log.gz 2024-08-17T02:25:53.895 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.29442.log 2024-08-17T02:25:53.895 INFO:teuthology.orchestra.run.smithi123.stderr:gzip /var/log/ceph/ceph.audit.log: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-08-17T02:25:53.897 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29424.log.gz 2024-08-17T02:25:53.897 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32203.log.gz 2024-08-17T02:25:53.897 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31779.log 2024-08-17T02:25:53.897 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30454.log 2024-08-17T02:25:53.897 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29854.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31356.log.gz 2024-08-17T02:25:53.897 INFO:teuthology.orchestra.run.smithi100.stderr:gzip /var/log/ceph/ceph-client.admin.31779.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29423.log 2024-08-17T02:25:53.898 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29854.log.gz 2024-08-17T02:25:53.898 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32176.log 2024-08-17T02:25:53.898 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30454.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31779.log.gz 2024-08-17T02:25:53.898 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-08-17T02:25:53.898 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29423.log: /var/log/ceph/ceph-client.admin.32176.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.30454.log.gz 2024-08-17T02:25:53.898 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30319.log 2024-08-17T02:25:53.898 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29429.log 2024-08-17T02:25:53.898 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29423.log.gz 2024-08-17T02:25:53.899 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32176.log.gz 2024-08-17T02:25:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29640.log 2024-08-17T02:25:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30319.log: /var/log/ceph/ceph-client.admin.29429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31113.log 2024-08-17T02:25:53.899 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29429.log.gz 2024-08-17T02:25:53.899 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30319.log.gz 2024-08-17T02:25:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30076.log 2024-08-17T02:25:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.3.29442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.29330.log 2024-08-17T02:25:53.904 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.3.29442.log.gz 2024-08-17T02:25:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24400.log 2024-08-17T02:25:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.1.29330.log: 93.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-17T02:25:53.904 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.1.29330.log.gz 2024-08-17T02:25:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24292.log 2024-08-17T02:25:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.24400.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.24400.log.gz 2024-08-17T02:25:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-08-17T02:25:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.24292.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.24292.log.gz 2024-08-17T02:25:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24239.log 2024-08-17T02:25:53.907 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.b.log: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-08-17T02:25:53.908 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-17T02:25:53.908 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.24239.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.24239.log.gz 2024-08-17T02:25:53.908 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24373.log 2024-08-17T02:25:53.913 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.log: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-17T02:25:53.914 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24267.log 2024-08-17T02:25:53.914 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.24373.log: 91.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-08-17T02:25:53.914 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.24373.log.gz 2024-08-17T02:25:53.914 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-08-17T02:25:53.914 INFO:teuthology.orchestra.run.smithi123.stderr:gzip /var/log/ceph/ceph-client.admin.24267.log: -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-08-17T02:25:53.914 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.4.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.24267.log.gz 2024-08-17T02:25:53.934 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29640.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29640.log.gz 2024-08-17T02:25:53.934 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33433.log 2024-08-17T02:25:53.934 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28741.log 2024-08-17T02:25:53.935 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31113.log.gz 2024-08-17T02:25:53.935 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30076.log: /var/log/ceph/ceph-client.admin.33433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31329.log 2024-08-17T02:25:53.935 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33433.log.gz 2024-08-17T02:25:53.935 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33171.log 2024-08-17T02:25:53.935 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.28741.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30076.log.gz 2024-08-17T02:25:53.935 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30669.log 2024-08-17T02:25:53.935 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31329.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28741.log.gz 2024-08-17T02:25:53.936 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29860.log 2024-08-17T02:25:53.936 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33171.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31329.log.gz 2024-08-17T02:25:53.936 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32041.log 2024-08-17T02:25:53.936 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30669.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33171.log.gz 2024-08-17T02:25:53.936 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28876.log 2024-08-17T02:25:53.936 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29860.log.gz 2024-08-17T02:25:53.936 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30669.log.gz 2024-08-17T02:25:53.936 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34525.log 2024-08-17T02:25:53.937 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31833.log 2024-08-17T02:25:53.937 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.28876.log: /var/log/ceph/ceph-client.admin.34525.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-08-17T02:25:53.937 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32041.log.gz 2024-08-17T02:25:53.937 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31833.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34525.log.gz 2024-08-17T02:25:53.937 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28876.log.gz 2024-08-17T02:25:53.937 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31833.log.gz 2024-08-17T02:25:53.937 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31725.log 2024-08-17T02:25:53.937 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.30621.log 2024-08-17T02:25:53.938 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.c.log: gzip /var/log/ceph/ceph-client.admin.31725.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29183.log 2024-08-17T02:25:53.938 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.30621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29129.log 2024-08-17T02:25:53.938 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31725.log.gz 2024-08-17T02:25:53.938 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28994.log 2024-08-17T02:25:53.938 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29183.log.gz 2024-08-17T02:25:53.938 INFO:teuthology.orchestra.run.smithi100.stderr: 90.6%/var/log/ceph/ceph-client.admin.29129.log: -- replaced with /var/log/ceph/ceph-client.0.30621.log.gz 2024-08-17T02:25:53.938 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29855.log 2024-08-17T02:25:53.939 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.28994.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29129.log.gz 2024-08-17T02:25:53.939 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32836.log 2024-08-17T02:25:53.939 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28994.log.gz 2024-08-17T02:25:53.939 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29855.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-08-17T02:25:53.939 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32836.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29855.log.gz 2024-08-17T02:25:53.939 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34471.log 2024-08-17T02:25:53.939 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.32836.log.gz 2024-08-17T02:25:53.939 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 /var/log/ceph/ceph-mds.a.log: --verbose -- /var/log/ceph/ceph-client.admin.33115.log 2024-08-17T02:25:53.940 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30292.log 2024-08-17T02:25:53.940 INFO:teuthology.orchestra.run.smithi100.stderr:gzip /var/log/ceph/ceph-client.admin.33115.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34228.log 2024-08-17T02:25:53.940 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34471.log.gz 2024-08-17T02:25:53.940 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33115.log.gz 2024-08-17T02:25:53.940 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33931.log 2024-08-17T02:25:53.940 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- 89.5%/var/log/ceph/ceph-client.admin.34228.log: /var/log/ceph/ceph-client.admin.32958.log 2024-08-17T02:25:53.940 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30292.log.gz 2024-08-17T02:25:53.940 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29853.log 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34228.log.gz 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32568.log 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32958.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33931.log.gz 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29156.log 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.32958.log.gz 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29853.log.gz 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31536.log 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32568.log: /var/log/ceph/ceph-client.admin.29156.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-08-17T02:25:53.941 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29156.log.gz 2024-08-17T02:25:53.942 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29415.log 2024-08-17T02:25:53.942 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31536.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33487.log 2024-08-17T02:25:53.942 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32568.log.gz/var/log/ceph/ceph-mgr.y.log: 2024-08-17T02:25:53.942 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29415.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31536.log.gz 2024-08-17T02:25:53.942 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31302.log 2024-08-17T02:25:53.942 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29415.log.gz 2024-08-17T02:25:53.942 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32460.log 2024-08-17T02:25:53.942 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29643.log 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31302.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33487.log.gz 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32460.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31302.log.gz 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32460.log.gz 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32014.log 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32541.log 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29643.log.gz 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33676.log 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33007.log 2024-08-17T02:25:53.943 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32014.log.gz 2024-08-17T02:25:53.944 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32541.log: /var/log/ceph/ceph-client.admin.33676.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32541.log.gz 2024-08-17T02:25:53.944 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33676.log.gz 2024-08-17T02:25:53.944 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-08-17T02:25:53.944 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31617.log 2024-08-17T02:25:53.944 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33088.log 2024-08-17T02:25:53.944 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30157.log 2024-08-17T02:25:53.944 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31617.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33007.log.gz 2024-08-17T02:25:53.944 INFO:teuthology.orchestra.run.smithi100.stderr:gzip /var/log/ceph/ceph-client.admin.33088.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32487.log 2024-08-17T02:25:53.945 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31617.log.gz 2024-08-17T02:25:53.945 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34174.log 2024-08-17T02:25:53.945 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30157.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33088.log.gz 2024-08-17T02:25:53.945 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32487.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30157.log.gz 2024-08-17T02:25:53.945 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34147.log 2024-08-17T02:25:53.945 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33514.log 2024-08-17T02:25:53.945 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34174.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32487.log.gz 2024-08-17T02:25:53.945 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32149.log 2024-08-17T02:25:53.946 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34174.log.gz 2024-08-17T02:25:53.946 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31644.log 2024-08-17T02:25:53.946 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34147.log.gz 2024-08-17T02:25:53.946 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30870.log 2024-08-17T02:25:53.946 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32149.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33514.log.gz 2024-08-17T02:25:53.946 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34039.log 2024-08-17T02:25:53.946 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32149.log.gz 2024-08-17T02:25:53.947 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24404.log 2024-08-17T02:25:53.947 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30870.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31644.log.gz 2024-08-17T02:25:53.947 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-17T02:25:53.947 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34039.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30870.log.gz 2024-08-17T02:25:53.947 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.24404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31698.log 2024-08-17T02:25:53.947 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34039.log.gz 2024-08-17T02:25:53.947 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.24404.log.gz 2024-08-17T02:25:53.947 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29645.log 2024-08-17T02:25:53.948 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30837.log 2024-08-17T02:25:53.948 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31698.log: /var/log/ceph/ceph-client.admin.29645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29861.log 2024-08-17T02:25:53.948 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31482.log 2024-08-17T02:25:53.948 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30837.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31698.log.gz 2024-08-17T02:25:53.948 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29645.log.gz 2024-08-17T02:25:53.948 INFO:teuthology.orchestra.run.smithi100.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-08-17T02:25:53.948 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29861.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30837.log.gz 2024-08-17T02:25:53.948 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph.audit.log.gz--verbose 2024-08-17T02:25:53.949 INFO:teuthology.orchestra.run.smithi100.stderr:-- /var/log/ceph/ceph-client.admin.29021.log 2024-08-17T02:25:53.949 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29861.log.gz 2024-08-17T02:25:53.949 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32676.log 2024-08-17T02:25:53.949 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33252.log 2024-08-17T02:25:53.949 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29021.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31482.log.gz 2024-08-17T02:25:53.949 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32902.log 2024-08-17T02:25:53.949 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.32676.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29021.log.gz 2024-08-17T02:25:53.949 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29102.log 2024-08-17T02:25:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33252.log: /var/log/ceph/ceph-client.admin.32902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33378.log 2024-08-17T02:25:53.950 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33252.log.gz 2024-08-17T02:25:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:gzip 88.4%-5 -- replaced with /var/log/ceph/ceph-client.admin.32902.log.gz--verbose 2024-08-17T02:25:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:-- /var/log/ceph/ceph-client.admin.31590.log 2024-08-17T02:25:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29102.log: /var/log/ceph/ceph-client.admin.33378.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29102.log.gz 2024-08-17T02:25:53.950 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33378.log.gz 2024-08-17T02:25:53.951 INFO:teuthology.orchestra.run.smithi100.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.32676.log.gz 2024-08-17T02:25:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29075.log 2024-08-17T02:25:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33306.log 2024-08-17T02:25:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32095.log 2024-08-17T02:25:53.951 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31590.log.gz 2024-08-17T02:25:53.951 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29075.log: gzip -5 /var/log/ceph/ceph-client.admin.33306.log: --verbose -- /var/log/ceph/ceph-client.admin.29833.log 2024-08-17T02:25:53.952 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29075.log.gz 2024-08-17T02:25:53.952 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32095.log: gzip 88.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.33306.log.gz--verbose 2024-08-17T02:25:53.952 INFO:teuthology.orchestra.run.smithi100.stderr:-- /var/log/ceph/ceph-client.admin.32755.log 2024-08-17T02:25:53.952 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32095.log.gz 2024-08-17T02:25:53.952 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33958.log 2024-08-17T02:25:53.952 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29833.log.gz 2024-08-17T02:25:53.952 INFO:teuthology.orchestra.run.smithi100.stderr:gzip /var/log/ceph/ceph-client.admin.32755.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31671.log 2024-08-17T02:25:53.952 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32755.log.gz 2024-08-17T02:25:53.953 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32068.log 2024-08-17T02:25:53.953 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33958.log: gzip /var/log/ceph/ceph-client.admin.31671.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34500.log 2024-08-17T02:25:53.953 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33958.log.gz 2024-08-17T02:25:53.953 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29638.log 2024-08-17T02:25:53.953 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31671.log.gz 2024-08-17T02:25:53.953 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32068.log.gz 2024-08-17T02:25:53.953 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31005.log 2024-08-17T02:25:53.953 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30978.log 2024-08-17T02:25:53.954 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29638.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34500.log.gz 2024-08-17T02:25:53.954 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31005.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29638.log.gz-5 2024-08-17T02:25:53.954 INFO:teuthology.orchestra.run.smithi100.stderr:--verbose -- /var/log/ceph/ceph-client.admin.32296.log 2024-08-17T02:25:53.954 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31005.log.gz 2024-08-17T02:25:53.954 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32433.log 2024-08-17T02:25:53.954 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30346.log 2024-08-17T02:25:53.954 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32296.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30978.log.gz 2024-08-17T02:25:53.954 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30924.log 2024-08-17T02:25:53.955 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32433.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32296.log.gz 2024-08-17T02:25:53.955 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29427.log 2024-08-17T02:25:53.955 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32433.log.gz 2024-08-17T02:25:53.955 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30184.log 2024-08-17T02:25:53.955 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30346.log.gz 2024-08-17T02:25:53.955 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30924.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30924.log.gz 2024-08-17T02:25:53.955 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32232.log 2024-08-17T02:25:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29427.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.19281.log 2024-08-17T02:25:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30184.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29427.log.gz 2024-08-17T02:25:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34255.log 2024-08-17T02:25:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32232.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30184.log.gz 2024-08-17T02:25:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.tmp-client.admin.19281.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-08-17T02:25:53.956 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32232.log.gz 2024-08-17T02:25:53.956 INFO:teuthology.orchestra.run.smithi100.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.19281.log.gz 2024-08-17T02:25:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28771.log 2024-08-17T02:25:53.957 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34255.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34255.log.gz 2024-08-17T02:25:53.957 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33460.log 2024-08-17T02:25:53.957 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34201.log 2024-08-17T02:25:53.957 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.28771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28835.log 2024-08-17T02:25:53.957 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33460.log: /var/log/ceph/ceph-client.admin.34201.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28771.log.gz 2024-08-17T02:25:53.957 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28938.log 2024-08-17T02:25:53.957 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33460.log.gz 2024-08-17T02:25:53.958 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.28835.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34201.log.gz 2024-08-17T02:25:53.958 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28641.log 2024-08-17T02:25:53.958 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- 88.3%/var/log/ceph/ceph-client.admin.30238.log 2024-08-17T02:25:53.958 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28835.log.gz 2024-08-17T02:25:53.958 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.28938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33622.log 2024-08-17T02:25:53.958 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.28641.log: /var/log/ceph/ceph-client.admin.30238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34336.log 2024-08-17T02:25:53.958 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28641.log.gz 2024-08-17T02:25:53.959 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28938.log.gz 2024-08-17T02:25:53.959 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30427.log 2024-08-17T02:25:53.959 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30238.log.gz 2024-08-17T02:25:53.959 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33622.log: /var/log/ceph/ceph-client.admin.34336.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33622.log.gz 2024-08-17T02:25:53.959 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34336.log.gz 2024-08-17T02:25:53.960 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.7.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-08-17T02:25:53.973 INFO:teuthology.orchestra.run.smithi123.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-08-17T02:25:53.987 INFO:teuthology.orchestra.run.smithi123.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-08-17T02:25:53.989 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33406.log 2024-08-17T02:25:53.989 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33541.log 2024-08-17T02:25:53.990 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30427.log.gz 2024-08-17T02:25:53.990 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32875.log 2024-08-17T02:25:53.990 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33406.log: /var/log/ceph/ceph-client.admin.33541.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33406.log.gz 2024-08-17T02:25:53.990 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31932.log 2024-08-17T02:25:53.990 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33541.log.gz 2024-08-17T02:25:53.990 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31248.log 2024-08-17T02:25:53.990 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29644.log 2024-08-17T02:25:53.991 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31932.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32875.log.gz 2024-08-17T02:25:53.991 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29637.log 2024-08-17T02:25:53.991 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31248.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31932.log.gz 2024-08-17T02:25:53.991 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34417.log 2024-08-17T02:25:53.991 INFO:teuthology.orchestra.run.smithi100.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31248.log.gz 2024-08-17T02:25:53.991 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29641.log 2024-08-17T02:25:53.991 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29644.log.gz 2024-08-17T02:25:53.992 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34417.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.29637.log.gz 2024-08-17T02:25:53.992 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34417.log.gz 2024-08-17T02:25:53.992 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31167.log 2024-08-17T02:25:53.992 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34066.log 2024-08-17T02:25:53.992 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33144.log 2024-08-17T02:25:53.992 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31167.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29641.log.gz 2024-08-17T02:25:53.992 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31086.log 2024-08-17T02:25:53.992 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34066.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31167.log.gz 2024-08-17T02:25:53.993 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24135.log 2024-08-17T02:25:53.993 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34066.log.gz 2024-08-17T02:25:53.993 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33144.log.gz 2024-08-17T02:25:53.993 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28675.log 2024-08-17T02:25:53.993 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31059.log 2024-08-17T02:25:53.993 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.24135.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31086.log.gz 2024-08-17T02:25:53.993 INFO:teuthology.orchestra.run.smithi100.stderr:gzip /var/log/ceph/ceph-client.admin.28675.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34093.log 2024-08-17T02:25:53.993 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.24135.log.gz 2024-08-17T02:25:53.994 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31059.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28675.log.gz 2024-08-17T02:25:53.994 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31059.log.gz 2024-08-17T02:25:53.994 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31032.log 2024-08-17T02:25:53.994 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.30763.log 2024-08-17T02:25:53.994 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31194.log 2024-08-17T02:25:53.994 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31032.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34093.log.gz 2024-08-17T02:25:53.994 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.30763.log: /var/log/ceph/ceph-client.admin.29858.log 2024-08-17T02:25:53.994 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31032.log.gz 2024-08-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 /var/log/ceph/ceph-client.admin.31194.log: --verbose -- /var/log/ceph/ceph-client.admin.33703.log 2024-08-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34282.log 2024-08-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29858.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31194.log.gz 2024-08-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi100.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.2.30763.log.gz 2024-08-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi100.stderr:gzip /var/log/ceph/ceph-client.admin.33703.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31383.log 2024-08-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29858.log.gz 2024-08-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29048.log 2024-08-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33703.log.gz 2024-08-17T02:25:53.996 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31383.log: /var/log/ceph/ceph-client.admin.30510.log 2024-08-17T02:25:53.996 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5%/var/log/ceph/ceph-client.admin.29048.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31383.log.gz 2024-08-17T02:25:53.996 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31275.log 2024-08-17T02:25:53.996 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34282.log.gz 2024-08-17T02:25:53.996 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32809.log 2024-08-17T02:25:53.996 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29048.log.gz 2024-08-17T02:25:53.996 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30510.log: /var/log/ceph/ceph-client.admin.31275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33333.log 2024-08-17T02:25:53.996 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30510.log.gz 2024-08-17T02:25:53.997 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31275.log.gz 2024-08-17T02:25:53.997 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30400.log 2024-08-17T02:25:53.997 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32809.log.gz 2024-08-17T02:25:53.997 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32350.log 2024-08-17T02:25:53.997 INFO:teuthology.orchestra.run.smithi100.stderr:gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33333.log.gz-5 2024-08-17T02:25:53.997 INFO:teuthology.orchestra.run.smithi100.stderr:--verbose -- /var/log/ceph/ceph-client.admin.33649.log 2024-08-17T02:25:53.997 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33595.log 2024-08-17T02:25:53.998 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32350.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30400.log.gz 2024-08-17T02:25:53.998 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31140.log 2024-08-17T02:25:53.998 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33649.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32350.log.gz 2024-08-17T02:25:53.998 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28903.log 2024-08-17T02:25:53.998 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33649.log.gz 2024-08-17T02:25:53.998 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29428.log 2024-08-17T02:25:53.998 INFO:teuthology.orchestra.run.smithi100.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-08-17T02:25:53.998 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33595.log.gz 2024-08-17T02:25:53.999 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31221.log 2024-08-17T02:25:53.999 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.28903.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31140.log.gz 2024-08-17T02:25:53.999 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33061.log 2024-08-17T02:25:53.999 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28903.log.gz 2024-08-17T02:25:53.999 INFO:teuthology.orchestra.run.smithi100.stderr:gzip 87.6%/var/log/ceph/ceph-client.admin.31221.log: -5 -- replaced with /var/log/ceph/ceph-client.admin.29428.log.gz--verbose 2024-08-17T02:25:53.999 INFO:teuthology.orchestra.run.smithi100.stderr:-- /var/log/ceph/ceph-client.admin.34444.log 2024-08-17T02:25:53.999 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29374.log 2024-08-17T02:25:53.999 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31221.log.gz 2024-08-17T02:25:54.000 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33061.log.gz 2024-08-17T02:25:54.000 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31806.log 2024-08-17T02:25:54.000 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29266.log 2024-08-17T02:25:54.000 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29374.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34444.log.gz 2024-08-17T02:25:54.000 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31806.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29374.log.gz 2024-08-17T02:25:54.000 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31806.log.gz 2024-08-17T02:25:54.000 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30049.log 2024-08-17T02:25:54.000 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31509.log 2024-08-17T02:25:54.001 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33857.log 2024-08-17T02:25:54.001 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30049.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29266.log.gz 2024-08-17T02:25:54.001 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33198.log 2024-08-17T02:25:54.001 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30049.log.gz 2024-08-17T02:25:54.001 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31509.log.gz 2024-08-17T02:25:54.001 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34012.log 2024-08-17T02:25:54.001 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33803.log 2024-08-17T02:25:54.001 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33198.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33857.log.gz 2024-08-17T02:25:54.002 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-17T02:25:54.002 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34012.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33198.log.gz 2024-08-17T02:25:54.002 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30130.log 2024-08-17T02:25:54.002 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33803.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34012.log.gz 2024-08-17T02:25:54.002 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33776.log 2024-08-17T02:25:54.002 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33803.log.gz 2024-08-17T02:25:54.002 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31410.log 2024-08-17T02:25:54.003 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30130.log.gz 2024-08-17T02:25:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.33776.log: -- /var/log/ceph/ceph-client.admin.31986.log 2024-08-17T02:25:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30951.log 2024-08-17T02:25:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31410.log: 84.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-17T02:25:54.003 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33776.log.gz 2024-08-17T02:25:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32728.log 2024-08-17T02:25:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31986.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31410.log.gz 2024-08-17T02:25:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34363.log 2024-08-17T02:25:54.004 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30951.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31986.log.gz 2024-08-17T02:25:54.004 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32261.log 2024-08-17T02:25:54.004 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32728.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30951.log.gz 2024-08-17T02:25:54.004 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32782.log 2024-08-17T02:25:54.004 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32728.log.gz 2024-08-17T02:25:54.004 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29859.log 2024-08-17T02:25:54.004 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32261.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34363.log.gz 2024-08-17T02:25:54.005 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32782.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-08-17T02:25:54.005 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32261.log.gz 2024-08-17T02:25:54.005 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30265.log 2024-08-17T02:25:54.005 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29859.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32782.log.gz 2024-08-17T02:25:54.005 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30897.log 2024-08-17T02:25:54.005 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29859.log.gz 2024-08-17T02:25:54.005 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33279.log 2024-08-17T02:25:54.005 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29347.log 2024-08-17T02:25:54.006 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30897.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30265.log.gz 2024-08-17T02:25:54.006 INFO:teuthology.orchestra.run.smithi100.stderr:gzip /var/log/ceph/ceph-client.admin.33279.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33904.log 2024-08-17T02:25:54.006 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30897.log.gz 2024-08-17T02:25:54.006 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33985.log 2024-08-17T02:25:54.006 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29347.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33279.log.gz 2024-08-17T02:25:54.006 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32323.log 2024-08-17T02:25:54.006 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33904.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29347.log.gz 2024-08-17T02:25:54.007 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32122.log 2024-08-17T02:25:54.007 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33985.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33904.log.gz 2024-08-17T02:25:54.007 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32323.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33985.log.gz 2024-08-17T02:25:54.007 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32514.log 2024-08-17T02:25:54.007 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32323.log.gz 2024-08-17T02:25:54.007 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32379.log /var/log/ceph/ceph-client.admin.32122.log: 2024-08-17T02:25:54.007 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32122.log.gz 2024-08-17T02:25:54.007 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30211.log 2024-08-17T02:25:54.008 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32619.log 2024-08-17T02:25:54.008 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32379.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32514.log.gz 2024-08-17T02:25:54.008 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29239.log 2024-08-17T02:25:54.008 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32379.log.gz 2024-08-17T02:25:54.008 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30211.log.gz 2024-08-17T02:25:54.008 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24270.log 2024-08-17T02:25:54.008 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32619.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32619.log.gz 2024-08-17T02:25:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33225.log 2024-08-17T02:25:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30810.log 2024-08-17T02:25:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.24270.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29239.log.gz 2024-08-17T02:25:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29293.log 2024-08-17T02:25:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33225.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.24270.log.gz 2024-08-17T02:25:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30373.log 2024-08-17T02:25:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30810.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33225.log.gz 2024-08-17T02:25:54.010 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34309.log /var/log/ceph/ceph-client.admin.29293.log: 2024-08-17T02:25:54.010 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30810.log.gz 2024-08-17T02:25:54.010 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.30373.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-08-17T02:25:54.010 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29293.log.gz 2024-08-17T02:25:54.010 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32929.log 2024-08-17T02:25:54.010 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34309.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30373.log.gz 2024-08-17T02:25:54.010 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 /var/log/ceph/ceph-osd.3.log: --verbose -- /var/log/ceph/ceph-client.admin.33034.log 2024-08-17T02:25:54.010 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 88.7%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.34309.log.gz/var/log/ceph/ceph-client.admin.34390.log 2024-08-17T02:25:54.011 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-08-17T02:25:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32929.log: /var/log/ceph/ceph-client.admin.33034.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32929.log.gz 2024-08-17T02:25:54.011 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33034.log.gz 2024-08-17T02:25:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33568.log 2024-08-17T02:25:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29426.log 2024-08-17T02:25:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31752.log 2024-08-17T02:25:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33568.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34390.log.gz 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29426.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33568.log.gz 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33830.log 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29320.log 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29426.log.gz 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31752.log: /var/log/ceph/ceph-client.admin.33830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28798.log 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31752.log.gz 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31563.log 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7%/var/log/ceph/ceph-client.admin.29320.log: -- replaced with /var/log/ceph/ceph-client.admin.33830.log.gz 2024-08-17T02:25:54.012 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29639.log 2024-08-17T02:25:54.013 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7%/var/log/ceph/ceph-client.admin.28798.log: -- replaced with /var/log/ceph/ceph-client.admin.29320.log.gz 2024-08-17T02:25:54.013 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31439.log 2024-08-17T02:25:54.013 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28798.log.gz 2024-08-17T02:25:54.013 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31563.log.gz 2024-08-17T02:25:54.013 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30103.log 2024-08-17T02:25:54.013 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33730.log 2024-08-17T02:25:54.013 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29639.log.gz 2024-08-17T02:25:54.013 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31439.log: /var/log/ceph/ceph-client.admin.30103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34120.log 2024-08-17T02:25:54.014 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31959.log 2024-08-17T02:25:54.014 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31439.log.gz 2024-08-17T02:25:54.014 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33730.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30103.log.gz 2024-08-17T02:25:54.014 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31868.log 2024-08-17T02:25:54.014 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34120.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33730.log.gzgzip 2024-08-17T02:25:54.014 INFO:teuthology.orchestra.run.smithi100.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.34552.log 2024-08-17T02:25:54.014 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34120.log.gz 2024-08-17T02:25:54.015 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32406.log 2024-08-17T02:25:54.015 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.31868.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-08-17T02:25:54.015 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31959.log.gz/var/log/ceph/ceph-client.admin.34552.log: 88.3% 2024-08-17T02:25:54.015 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31868.log.gz 2024-08-17T02:25:54.015 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34552.log.gz 2024-08-17T02:25:54.015 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.32406.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32406.log.gz 2024-08-17T02:25:54.015 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29425.log 2024-08-17T02:25:54.015 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28967.log 2024-08-17T02:25:54.016 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29210.log 2024-08-17T02:25:54.016 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29425.log: /var/log/ceph/ceph-client.admin.28967.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29425.log.gz 2024-08-17T02:25:54.016 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28967.log.gz 2024-08-17T02:25:54.016 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.29210.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29210.log.gz 2024-08-17T02:25:54.068 INFO:teuthology.orchestra.run.smithi123.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-08-17T02:25:54.088 INFO:teuthology.orchestra.run.smithi100.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-08-17T02:25:54.104 INFO:teuthology.orchestra.run.smithi100.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-08-17T02:25:54.104 INFO:teuthology.orchestra.run.smithi123.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-08-17T02:25:54.123 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-08-17T02:25:54.124 INFO:teuthology.orchestra.run.smithi123.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-08-17T02:25:54.124 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-08-17T02:25:54.126 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-08-17T02:25:54.126 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m0.259s 2024-08-17T02:25:54.127 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m1.091s 2024-08-17T02:25:54.127 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m0.055s 2024-08-17T02:25:54.145 INFO:teuthology.orchestra.run.smithi100.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-08-17T02:25:54.188 INFO:teuthology.orchestra.run.smithi100.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-08-17T02:25:54.189 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-08-17T02:25:54.189 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m0.324s 2024-08-17T02:25:54.189 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m1.190s 2024-08-17T02:25:54.190 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m0.087s 2024-08-17T02:25:54.190 INFO:tasks.ceph:Archiving logs... 2024-08-17T02:25:54.190 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/remote/smithi100/log 2024-08-17T02:25:54.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-17T02:25:54.551 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/remote/smithi123/log 2024-08-17T02:25:54.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-17T02:25:54.703 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-17T02:25:54.727 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-17T02:25:54.728 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-17T02:25:54.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-17T02:25:54.792 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-17T02:25:54.800 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-17T02:25:54.801 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-17T02:25:54.803 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-17T02:25:54.825 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-08-17T02:25:54.825 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-08-17T02:25:54.825 INFO:teuthology.orchestra.run.smithi100.stdout: hv01.front.sepi 67.205.162.81 3 u 49 64 177 0.108 0.350 0.830 2024-08-17T02:25:54.825 INFO:teuthology.orchestra.run.smithi100.stdout:*hv02.front.sepi 63.231.80.2 3 u 51 64 177 0.086 0.447 0.811 2024-08-17T02:25:54.826 INFO:teuthology.orchestra.run.smithi100.stdout: hv03.front.sepi 74.6.168.72 3 u 45 64 177 0.080 -1.184 0.738 2024-08-17T02:25:54.826 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:25:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: remote refid st t when poll reach delay offset jitter 2024-08-17T02:25:54.828 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================== 2024-08-17T02:25:54.828 INFO:teuthology.orchestra.run.smithi123.stdout:+hv01.front.sepi 67.205.162.81 3 u 45 64 177 0.089 -0.157 0.276 2024-08-17T02:25:54.828 INFO:teuthology.orchestra.run.smithi123.stdout:+hv02.front.sepi 63.231.80.2 3 u 46 64 177 0.077 -0.002 0.244 2024-08-17T02:25:54.829 INFO:teuthology.orchestra.run.smithi123.stdout:*hv03.front.sepi 74.6.168.72 3 u 54 64 177 0.075 -0.516 0.345 2024-08-17T02:25:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-17T02:25:54.830 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-17T02:25:54.839 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-17T02:25:54.839 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-17T02:25:54.847 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-17T02:25:54.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-17T02:25:54.864 INFO:teuthology.task.internal:Duration was 784.642622 seconds 2024-08-17T02:25:54.865 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-17T02:25:54.916 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-17T02:25:54.916 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-17T02:25:54.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-17T02:25:54.975 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-17T02:25:54.975 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-08-17T02:25:54.975 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-17T02:25:54.986 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-08-17T02:25:54.987 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-17T02:25:55.000 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-17T02:25:55.000 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-17T02:25:55.002 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-17T02:25:55.039 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-17T02:25:55.040 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-17T02:25:55.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-17T02:25:55.161 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-17T02:25:55.171 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-17T02:25:55.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-17T02:25:55.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-17T02:25:55.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-17T02:25:55.214 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-17T02:25:55.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-17T02:25:55.229 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-08-17T02:25:55.229 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-08-17T02:25:55.246 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-17T02:25:55.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:25:55.293 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-17T02:25:55.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:25:55.299 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-17T02:25:55.308 INFO:teuthology.task.internal:Transferring archived files... 2024-08-17T02:25:55.309 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/remote/smithi100 2024-08-17T02:25:55.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-17T02:25:55.337 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-17_01:02:08-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858246/remote/smithi123 2024-08-17T02:25:55.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-17T02:25:55.369 INFO:teuthology.task.internal:Removing archive directory... 2024-08-17T02:25:55.369 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-17T02:25:55.371 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-17T02:25:55.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-17T02:25:55.388 INFO:teuthology.task.internal:Not uploading archives. 2024-08-17T02:25:55.388 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-17T02:25:55.397 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-17T02:25:55.398 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-17T02:25:55.417 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-17T02:25:55.423 INFO:teuthology.orchestra.run.smithi100.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 17 02:25 /home/ubuntu/cephtest 2024-08-17T02:25:55.426 INFO:teuthology.orchestra.run.smithi123.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 17 02:25 /home/ubuntu/cephtest 2024-08-17T02:25:55.427 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-17T02:25:55.438 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-17T02:25:55.463 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} duration: 784.6426219940186 failure_reason: 'Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=059d8cab2ce740ccb5704f2ae520bc53 status: fail success: false 2024-08-17T02:25:55.464 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-17T02:25:55.539 INFO:teuthology.run:FAIL